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Based on an in-siiu electrical alterable nonvolatile scmiconducfbr memory, the 
MNOS transistor, an adaptive mtcrconneci is developed to implement wafer-scale in- 
tegration. Experimental validation of the interconnect is reponed. The interconnea 
concept is further extended to the design of semiconductor mass memory and the design 
of an adapn\"e voter to implement fault tolerant systems. 



§1. Introductioo 

Adaptive Wafer Scale Integration (AWSI) is 
a concept which originated in late 1971 at 
Aciron,* a division of McDonnell Douglas 
Corporation.*' It employs electrically alterable, 
nonvolatile interconnect controller circuits 
processed into a semiconductor wafer to con- 
nect "arrays" of interconnected, operable 
circuits (also processed into the wafer) to a bus 
structure deposited on the wafer between the 
arrays. With this approach. AWSI can have 
important advantages relative to other high 
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density electronic circuit approaches. Such 
advantages potentially include: repeated elec- 
tronic reconfigurability of interconnected cir- 
cuits, compatibility with a wide variety of 
semiconductor processes, ability to select for 
yield, improved reliability, self-healing and 
fault tolerance; all plus reduction in size, 
weight, and cost. 

Figure I graphically illustrates the AWSI 
concept in contrast with conventional inte- 
grated circuit (IC) technology. In both tech- 
nologies the "chips" or "dice" ^called arrays 
in AWSI) arc probed immediately after the 
completion of wafer processing to test for 
operability. In conventional IC technology, 
those dice which do not pass the probe test 
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arc marked and discarded when the wafer is 
diced into individual chips. 

In AWSI, the die array are all connected via 
system bus. The address of these arrays that 
pass the probe test are stored in nonvolatile 
memory. Operable arrays which are initially 
used in system mechanization are accessible 
either via indirect addressing or via associative 
decoder addressing. The remainder of the 
operable arrays are reserved as spares to be 
connected into the bus structure as replace- 
ments for operating arrays which subsequently 
may be shown by built-in tests to have become 
defective. Alternatively, the spares can be 
later used for system reconfiguration, dependent 
on application requirements. 

The use of a repeatedly alterable, electrical 
interconnect, processed into the wafer provides 
the principal advantages of AWSI compared to 
other whole-wafer technologies.^"** With 
AWSI interconnect 

a. Any array ma> be connected to. or dis- 
connected from. . the structure at any 
time; connections include both signals 
and power. 

b. Neither nonopcrable nor nonoperating 
arrays draw power. 

c. Neither special contact masks, metaliza- 
tion masks nor fusible links are required 
to connect arrays to bus lines. Condi- 
tional interconnect is accomplished elec- 
tronically instead of via nonalterable 
mechanical means. 

d. Spare operable arrays arc stored on the 
wafer and connected to the bus structure 
whenever built-in tests reveal that one of 
the active arrays has failed and must be 
replaced. 

e. Reconfiguration may be achieved within 
limits established via the system architec- 
ture by reconnecting arrays to the bus. 

§2. Basis for AWSI 

The technological basis for the AWSr inter- 
connect is the nonvolatile semiconductor 
memory transistor, the MNOS (mctal-niiride- 
oxidc-semiconducior) transistor.^*** 

The MNOS transistor is unique among solid- 
state electronic devices in that it permits stored 
data alterability simultaneously with stored data 
nonvolaiility.***"^^' It also provides a small- 
area structure and fabrication compatibility 



with high density, integrated circuit technol- 
ogies. 

The MNOS transistor is a close relative of 
the conventional metal-oxide-semiconductor 
(MOS) transistor in that the usual layer of gate 
oxide is replaced by a 500 A layer of silicon 
nitride over a less than 20 A thick layer of 
silicon dioxide. The application of a moderately- 
high vblugc (approximately 25 volts) to the 
gate electrode of this transistor causes the thin 
silicon dioxide layer to become conductive (or 
to permit charge carriers to tunnel through it). 
Charge carriers may then pass between the 
silicon and charge-carrier traps located near 
the silicon-nitride, silicon-dioxide interface. The 
presence of trapped charge carriers at this 
interface modifies the gate voltage which con- 
trols passage of charge carriers from source to 
drain in the conventional operation of the 
transistor. The MNOS transistor is then said 
to have an **ofT" and an *'on" states which 
depend on the concentration and the polarity 
of the trapped charge carriers. 

The MNOS transistors, utilized in a circuit, 
effectively became the means with which elec- 
trically alterable interconnection can be im- 
plemented in place of hardwired interconnec- 
tions. The alterable interconnect can be so 
utilized because the interconnection has a 
nonvolatile memory for sutus control so that 
the state of interconnected machine is static as 
if hardwired. Yet it is alterable, (that is adap- 
tive), via control signals by circuit means. 

The adaptive feature of the interconnect 
makes it possible to implement sparing to 
produce high reliability memory systems. A 
minimum number of spare memory circuits is 
set aside on the wafer. Upon detection of 
failure, the failed component is disconnected 
and the spare is connected to the system bus. 
Utilizing sparing redundancy, high system 
reliability is accomplished with minimum hard- 
ware cost and little increase in parts count. An 
illustrative example is given in Fig. 2. Assume 
16 memory circuits interconnected to form a 
high reliability memory subsystem, assigning 
same failure rates to equivalent-function units, 
it can be seen that using two spares for a 16 unit 
serial memory system, an order of magnitude 
improvement in calculated failure rate is 
achieved over that utilizing the dual-redundant 
approach. The improvement is impressive since 
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Fig. 3. Use of AVSSl to imerconnea a shift register 
string. 

the sparing reduncancy is realized with reduced 
hardware requirements and minimum operating 
power. 

As an introduction to the AWSI concepts in 
concrete physical terms, .we can use the example 
of an early feasibility demonstration vehicle, the 
reconfigurable memory wafer circuits, RM WC- 
I. The interconnea is composed of an electri- 
cally alterable, nonvolatile, MNOS memory 
cell and a standard logic or buffer gate NMOS 
transistor circuit or bus controller. The bus 
controller is enabled by the enabling signal 
which is conditional on the stored interconnect 
status of the memory cell. The bus controller 
connects or gates the power, clock and data 
signals from the interconnecting bus to the 
engaged circuit, in this case, a shift register. The 
relationship of the MNOS memory cell, the 
bus controller and the shift register in the 
AWSI implemenuiion is shown in Fig. 3. 




Fig. 4. Use of AWSI to interconnect shift register 
strings on a wafer, (the RMWC-1). 

This particular circuit has been fabricated 
and integrated on a wafer, Fig. 4; the data path 
organization is illustrated in Fig. 5. It may be 
seen that a shift register in a given horizontal 
string may be cither conneaed in series with 
the data line or bypassed. Each repeated com* 
bination is designated as an array. The metali- 
zation panem for this wafer circuit is shown in 
Fig. 6. 
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Fig. 5. Dau path organization of RMWOl. 
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Ftg. 6. The metaJizauoo pattern of RMWC-1. 



In ihc nonvolatile mass memory system, the storage array in a mass memory system results 

application of AWSI is further extended. On- in very low system powwer as compared with 

ihc-wafcr interconnect is mechanized such other memories implemented with RAM's or 

that small single-chip sequential access memory CCD*s. Power need be applied only to those 

arrays (with memory storage capacity in the storage arrays accessed, with no standby power 

range of 4 K to 16 K. bits per memory array) necessary to maintain data in unaccessed 

are interconnected to result in a large capacity storage arrays. Additionally, with the use of 

store (on the order of 10* bits) in a single 4 inch AWSI sparing redundancy, the mass memory 

wafer. The use of nonvolatile memories as the achieves system reliability at low cost. 
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§3. Memory System Hardware 

The memory system can be divided into wafer 
memory modules and the off-wafer circuitry, 
Fig. 7. The maia components of the off-wafer 
circuitry are the systen: controller and the error 
detection/correction c:rc&itry. The wafer mem- 
ory modules consist c: memory wafers, which 
in turn consist of the channel controllers (CQ, 
the array controllers ( AQ, the memory arrays 
(MA), and the buses which connect them. Fig. 8. 

The off-wafer system controller supplies the 
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intelligence of the system. Not only does the 
system controller interpret the system com- 
mands and supply detailed instruction sequenc- 
ing to the wafers, it also monitors the operation 
of all the system components. Should a fault be 
detected, the system controller will execute a 
built-in-test to isolate the fault, and then 
reconfigure the system to remove the fault. 

The off-wafer error detection/correction cir- 
cuitry performs tfie encoding/deconding and 
error correction on the system data. In conjunc- 
tion with the system buffer, the error detection/ 
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Fig. 7. Memory system organiiation of a gigabit mass memory. 
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Fig. 9. MAW-1 overview: associative decoder. SAM with internal pads, and maao-bus. 
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correction circuitry also performs a bit trans- 
formation on the encoded data to ensure that 
each bit of the dau word is stored in an 
independent memory array. 

The channel and array controllers resident in 
the memory wafcn implement the memory 
addressing technique by seleaively turning on 
power, switching data and control information 
to the intended group of memory arrays. This 
addressing struaure minimizes the power con- 
sumption of the memory wafers. The memory 
wafers constitute the bulk of the memory system 
and contain many of the unique features of the 
system. 

§4. The Reconfigurable Memory Wafer 

As a fint step towards the implementation of 
a memory wafer for mass memory application, a 
reconfigurable memory wafer (MAW-l) was 
developed and studied. The MAW- 1 is com- 
posed of interconnected memory arrays for the 
purpose of evaluating the circuits necessary to 
accomplish power switching, address storage 
and decode address control for interconnect 
reconfiguration; Figure 9 is an overview of the 
MAW-l which shows the physical arrangement 
of the device. Figure 10 is a photomicrograph 
of the MAW-l circuit array illustrating also the 
wafer level interconnect, or the macrobus. 

A 4096 bit nonvolatile sequential access 
memory (SAM) was the basic memory storage 
array. Several features of the SAM array 
uniquely esublish its application in an AWSI 
memory: (1) Memory storage is nonvolatile, 
based on n channel MNOS storage. (2) Dau 
input is high impedance and buffered with an 
input latch; data output is tri-stagc. and 
buffered for bus driving. (3) To reduce inter- 
connea bus lines, access within the SAM 
storage array is sequenced internally, timing 
input is resiriacd to a single clock line, and 
operation instruaions are decoded internally. 
(4) For multiplexed operation in a bus inter- 
connected system, a chip select control and a 
flag bit are provided. Table I summarizes the 
SAM device characteristics. 

The mechanization of on-t he- wafer inter- 
connect is accomplished with the development 
of an associative interconnect. The associative 
interconnect consists of six major components: 

(1) The macrobus which is the system bus, 

(2) power switches, (3) power connect/dis- 



Table I. SAM characteristics 
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connect logic, (4) nonvolatile address memory, 
(5) address memory control, (6) address com- 
parators. 

The associative interconnect was tested for 
(I) address memory storage and pattern 
insensitivity. (2) power-on control to the SAM 
by associative decoder address match and mis- 
match, (3) power turn-off of the SAM by 
address match, (4) permanent programmable 
match disable to eliminate bad decoder circuits 
and/or SAM's, (5) power response time of 
5 Msec. With the associative interconnected 
tested for operation, the MAW-l was tested 
for accomplishment of (1) selective power 
switching via the macrobus, (2) selective control 
of SAM circuit operation (read, write and erase) 
from the macrobus, and (3) lockout of defective 
SAM'S. The successful completion of the 
MAW-l tests led to the design of a megabit 
wafer memory for a prototype mass memory. 

§5. Fault Tolerant Systems 

Other than memory applications, the unique 
architectural structures available via AWSI are 
applicable to general purpose fault-tolerant 
computer systems. Some of these structures arc: 
on-wafer spares, nonvolatile memory switches 
to maintain on-wafer configuration, associative 
address decoders to provide hard and soft 
addressing capabilities, nonvolatile memory 
configuration and status tables to preserve the 
state of a system during power interrupu and 
failures, and small highly testable components 
which arc combined to form a large, on-wafer 
fault-tolerant system. 
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Fig. 11. C. vmp bus level configuration. 
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Let us consider an adaptive C.vmp voter 
circuit for application in a fault tolerant system. 
The C.vmp is developed at Carnegie-Mellon 
University, and stands for Computer (voted 
multiprocessor). ^^•^^> It was designed specifi- 
cally as a machine for the study of fault 
tolerance and the efEects of faults, both transient 
and permanent, on the system. C.vmp, shown 
in Fig. 11, uses three DEC LSMI computers 
organized in a TMR configuration. The ma- 
jority voting of the TMR system occurs at the 
interface of the processors to the system buses 
so the voter is between the processors, and the 
memories and peripherals. This allows voting 
to be done every bus cycle. Program studies 
have shown that every memory clement of the 
system is voted upon regularly, except the 
stack pointer, without any special software. 

Two of the major design considerations were 
that fault tolerance be carried out in real time, 



and that it be transparent to the software; 
this dicuted a TMR configuration. Another 
design consideration was that the system user 
be able to dynamically trade fault tolerance for 
throughput This characteristic resulted in a 
system which can operate in three modes: a 
loosely-coupled three computer mode, a fault- 
tolerant TMR mode, and the broadcast mode 
which allows a nonredundant peripheral to 
communicate.with all three of the processors. 
Each of these is controlled by the state of the 
voter. 

With the use of MNOS nonvolatile storage, 
an adaptive C.vmp voter is implemented, Fig. 
12. The flexibility of the adaptive C.vmp voter, 
is apparent from the wide variety of voting 
networks which can be constructed with differ- 
ent arrangements of the voter. The voters caa 
be cascaded together to form either a uni- 
directional or bidirectional voting network of 
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any width. (Here uaidirectional and bidirec- 
tional strictly apply to the voting, the data 
path itself is always bidirectiooal through the 
voting circuit.) An 8-bit unidirectional voting 
network is illustrated in Fig. 13, where two 4.bit 
voters supply the 8-bit voted data path and two 
additional voters provide the voted control 
inputs. A bidirectional 8-bit voting network 
would simply consist of two unidirectional 
networks connected together, with one being 
the mirror image of the other. The voting on 
the control inputs can also be configured to 
the desired application. Each voter's power 
switch and mode control inputs can be con- 
nected to a single set of voting inputs, a multiple 
set of separate voting inputs, or any combina- 
tion. The adaptive voter circuitry, can be used 
to implement the hybrid redundancy through 
the use of MNOS controlled power switches. 
As illustrated in Fig. 13, whenever one of the 
microprocessor fails, its power is removed and 
a spare unit is powered up to replace it. The 
nonvolatile M NOS status memory reuins 
the proper configuration of functional micro- 
processors even after a system power down. 
Should enough of the microprocessors fail so 
that no spares remain, the voter circuit is 
switched to multiprocessor mode, so that the 
system operates with a single microprocessor 
plus a spare thus circumventing the coverage 
probelm of TMR systems when caustrophic 
failure resulting in only two operating units 
out of three units necessary for majority voting. 

§6. Concluding Remarks 
Based on a nonvolatile semiconductor 



memory device, the MNOS transistor, an 
interconnect technology is developed. The 
application of the interconnect technology, in 
connection with the memory transistor, results 
in electronic systems which can be characterized 
as highly reliable and fault tolerant. Further 
development of the MNOS based adaptive 
circuits and interconnect undoubtedly will lead 
to self-organizing maclunes with artificial 
intelligence. 
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Section 1 

MASS K€KJRY SYSTEM DESCRIPTION (PART 1) 



1.0 INTRODUCTION 



The next generation mass memory system snould and can possess ail of tne fol- 
lowing cnaracteristics: 



• Ljow power consLfTption 

• System level nonvolatility 

§ Hig^ data rate (eouivalent to computer main memory) 

• Hign reliability 

• ADility to reconfigure and/or gracefully degrade 

• Fault tolerance 

• Functional modularity and expandability 

• Ljow weignt 

• Small volone 

• Low cost 



This report describes a mass memory system that will meet all of these demands. 
Initial efforts r«ve defined the technology tnat will si<)port tne development 
of a lignt-weiytt, small volane, low power, 0,5 x lo'-bit semiconckictor memory. 
Che application could be a niy»ly-reliable replacement for presently employed 

magnetic tape recorders. 

Two higily innovative semiconductor technologies are proposed to achieve this 
goal; adaptive wafer scale integration and an n-channel metal-nitride-oxide- 
semiconductor (KNOS) transistor memory device. These will satisfy tne inmediate 
objective to develop a cost-effective functional substitute for tne magnetic 
tape recorder. One or both of these technologies can be the basis not only for 
'Tiemory systems, but also for computers or encryption equipment. They are gen- 
erally applicable to any complex information processing and storage reouirement 
where cost, reliability, size, and/or weight are important. 
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i.i ADAPTIVE KtfVER SCALE INTEGRATION (WSI) 

Hig^ syston relladility, computed to be greater tnan a 0.95 prodaoility of sys- 
tem operation over 5 years, is obtained by using a new and LTioue integrated 
circuit tecnnoiogy called adaptive wafer scale integration (AWSI). It permits 
large nortoers of memory arrays to be interconnected (repeatedly) by electri- 
cally alterable, nonvolatile, latcning interconnect circuits, Botn memory 
arrays and latcning interconnect circuits are fabricated on the same semicon- 
ductor wafer substrate tnrougn a series of pnotolitnograpnic process steps, 
Tbe latcning interconnect circuits proviae control for address, power and data 
bus lines fabricated by metallization steps, Tbe use of tncse interconnect 
circuits results in a minimization of bonding pads and in a retfcjction of bus 
line loading wnicn results in lower power reouirement. Because tnese intercon- 
nect circuits are made as a part of tne semiconductor wafer fabrication pro- 
cess, tne connection of logic function and memory circuit arrays to tne buses 
on tne wafer eliminates large nurtiers of potentially unreliable electrical con- 
nection Donos, often tne most unreliable part of an LSI circuit, plus substan- 
tial amounts of manufacturing labor and material. 

under tne control of peripheral error detection/correction and built-in test 
circuits, tne interconnect circuits enable malfunctioning arrays to be replaced 
autoTBtically by spare arrays, wnicn are available on the wafer. Since tne 
interconnects are electrically alterable, tnis sparing process can be carried 
out repeatedly wnen required. Because tney are nonvolatile, tne interconnect 
configuration is retained over periods of power down or power interruption. 

Tne basic intercofwct circuit, wnicn nas been developed in an independent 
researcn and development program, is a standard logic or buffer gate ^M3S tran- 
sistor circuit tnat is enabled by an electrically-alterable, nonvolatile, MNOS 
frenory cell. The use of this circuit to interconnect a shift register to a 
data Dus is srown in figure 1-1. This particular example circuit nas been fab- 
ricated and integrated on a wafer where a shift register in a given horizontal 
strirq may be either connected in series with the data line or bypassed (to 
sinvjiate the removal of a faulty register from the bus). 



1-2 



vo I Lint ill 



'00 



CLOCK 
DATA IN 



DATA OUT ^ 



MNOS 
MEMORY CELL 



1 



INTERCONNEa 

BUS 
CONTROLLER 



SHIFT 
REGISTER 

DATA IN 
DATA OUT 



INTCKCOMta 








txcctrrivc 









7v 



IHTIHeOttRICT 
tXiCUTlVl 



tmrr 




Hrrmcomita 








IKieUTIVl 











•C) 



$HlfT 
NCGISTIM 



Figure i-1. Use of AWSI to Interconnect a Shift Register String 

The mass memory system , oescribed in detail in following sections, concines 
tne AWSI interconnect tecnnology witn a nonvolatile n-cnannel MNOS transistor 
memory device. 

Qrt-tne-wafer interconnect is mecnanized sucn tnat small seouential access men>* 
ory arrays witn a memory storage capacity in tne range of Ak to 8k bits per 
memory array are interconnected to obtain a large capacity store approximately 
0.5 X 10^ bits in a single wafer. 



1.2 FRELMNARY SYSTEM TOIIFICATION (DESIGN QQALS) 



The preliminary specifications for a 0.5 x 10^ -bit mass memory system wnlcn are 
presented in table 1^1, are for a seouential access mass memory finctionally 
intended as a magnetic tape recorder replacement. The memory system design 
pniiosopny is governed by tne following major factors. The storage level of 
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TflaLE i-i. RELIMINARY SPECIFICATION TOR lO^-BIT flWSI l'€MDRY 



TEChNXOGY 
SYSTEM 

Nonvolatile self-repairing, electrically-alterable interconnect and unpo«»- 

ered redundant sparing. 

CDKPOhENT 

Nonvolatile memory (hM4}/ sequential access memory arrays-using wafer scale 

integration. 

PROCESS 

MOS/hMDS 

3-incn wafer (initial development) 

4.incn wafer (engineering model) 
STORAGE CPPACITY 

(Wafer level) 0.5 Msits U.incn wafers) 

(System level) 0.5 x 10 'data bits 
FEATURES 

Nonvolatile storage 
Self-repairing 

File organized (256 Kbits/file; 2048 files total) 
Power switcning 
Error detection/correction 
I/O SERIAL DATA RATE 

5 m/s 

POV€R-ON TIf€ 

1 s exclusive of power supply 
FILE ACCESS TI^€ 

1 ms random access 

R3V€R 

Total 

ftak Average R)«er 
^ Rower R)wer Qonsuned 

READ 27.5 W 25.7 W 3.8 x 10"* Whr/fUe 
ERASE 37.0 W 33.4 W 5.9 x 10*5 vir»r/file 
WRITE 38.4 W 35.9 W 5.3 x 10"* y*^r/file 

RELIABILITY 

5 year reliaoility of 0.95 
vaLL^€ 

2,200 cu incnes 
WEIGHT 

60 pounds 



1-4 



(•EX15421 



voiLfne III 



0.5 X iO^-oits and tne aata rate of 5 Mb/s are selected as typical nonsers for 
a recorder application. The system reliability is specified as greater tnan 
0.95 for five years of unattended operation; tne system design employs error 
detection and correction circuitry and unpowered spare memory elements in order 
to perform irv-situ repair of tne memory system to tnaintain full capability for 
tne five-year period. This tecnnioue for nign reliability is extended beyond 
tne meiory arrays tnemselves to provide built-in test and sparing for otner 
parts of tne mass memory system wnicn include tne system controller and tne 
basic error detection circuitry. 

Tne marory capacity of 0.5 x lo' bits of data, witn a i2X growtn fdr error de- 
tection and correction coding, results in a total storage reouirement of ap- 
proxirately 0.6 x lo' bits of information. The total system reliability can be 
acyiieved witn a 5 percent sparing level. Thus, tne total memory nust include 
storage capacity for data, code bits, and spare memory elements. The file size 
of 256 kbits is crosen as a representative memory reouirement. The basic ele- 
ment of tne mass memory system is the memory wafer, wnicn is organized in four 
separately operated quadrants. No restrictions are placed on similarity of eacn 
Quadrant in order to provide tne most versatile system design and td miniinize 
tne cost of fabrication. 

Tne reouirerent for minimization of botn average and peak power results in 
several canpranises in the system design, most notably a limitation on the 
speed of tne system. However, the overall speed of tne system can be increased 
witn relaxation of tne power requirements. Articular attention is given to 
power seouencing of tne components to minindze total power coosonption. Much 
of tne system ovemcad is allocated to tnis power sequencing. 

The menory system organization assines tnat tne memory is written into in one 
time Interval and read in anotner. no provision is made in tne current system 
to read one section of tne memory wnile writing in anotner section. However, 
TOdifications to the system controller can be made to allow sucn operation. In 
tnis respect, tne operation is siMlar to that of a conventional tape recorder. 
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Tne error-correcticx^ tecnnioue is preaicatea on tne detection of errors during 
operation and tnen performing built-in test and reconfiguration off-line, 
i.e, f wnen tne Dasic memory system is not in use. 

1.3 Sim«Y 

Tnis report presents tne results of the system design activity. Section 2.0 
is tne system nardware arcnitecture and includes brief descriptions of tne mass 
mmry systan components, ircluding tne wafer and its circuit elements and tne 
controllers external to tne basic wafer organization. The hardware description 
of section 3.0 presents the details of tne mass memory system components. 
Section 4.0 presents tne details of tne memory system operation, concentrating 
on tne addressing procedures and tne memory wafer command seouencing. The sys- 
tem software of section 5.0 presents further details of memory operation, 
including the procedures for built-in^test and reconfiguration. Section 6.0 
describes tne procedures to be used in testing tne memory wafers, including tne 
wafer acceptaoility criteria. The yield of the memory wafers is described in 
section 7.0. Section 8.0 discusses the reliability of tne memory system, and 
section 9.0 details tne memory system packaging, including the memory wafer 
packaging. Sections 1 through S are presented in tnis volune. Sections 6 
through 9 are presented in Volone IV, Mass Memory System Description (Part 2). 
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Section 2 
SYSTEM HflRD»(ARE AflCHITECTURE 

2.0 mSl K^S N6MDRY AlCHITECTURE 

Tne oesiyt approacn for tne mass memory system nas eirpnasized tne minimiza- 
tion of system power consurption wnile meeting tne operaUonal specifications 
described previously. The major factors impacting tne system design include 
tne mininum addressable file size, tne mininum bit error rate in tne data, and 
tne reouirwent to provide recovery of tne system after failure of any single 
elBient. The total capacity of tne memory nas been specified as 0.5 x lo' data 
bits. The marory elements nave been organized Into wafer Quadrants in order 
to provide rmxiitum versatility and to allow tne maxlnun nuitoer of tne processed 
wafers to be etrployed in tne system. The organization does not require a fixed 
minimum nwnory capacity per wafer or per quadrant, but ratner tne system is 
organized to acconnodate varying capacities on tne wafers wnicn make up tne 
memory system. 

Tne reconfigurability of tne memoxy to acnieve tne 0.95 reliability for 5 years 
will oe acnieved via single-bit error correctlon/double-dit error detection, 
on-line reconfiguration, and built-in test (BIT) programs. Replacement of 
failed circuits will be accompiisned wnen tne memory, is not being actively 
addressed. 

utilization of tne fuictional circuits wnile avoiding tne malfixictional ones 
nas to be accompiisned in sucn a manner tnat tne overnead circuits constitute 
a reasonable fraction of tne total in order tnat tne desirable memory cnarac- 
teristics are effectively preserved and tne resulting system remains manage- 
aole. Two main tecnniques are employed: a partyline bus concept wnere units 
on tne bus respond upon recognition of tneir unique code (for tnat particular 
bus) ; and on-wafer mapping wnere an off-wafer controller need not be concerned 
as to wnicn circuits are functional or malfunctional during data I/O modes. 



2-1 



LEX15424 



voione III 



Tne on-wafer ccyitroi circuits are designed to take maxinun advantage of and be 
consistent witn tne process used in tne memory array fabrication. Tne implica- 
tions of tnis are slow circuit operation with no ni^speed clocks, a mininun 
^Tount of seouential logic , a liberal usage of nonvolatile memory storage 
(nigniy structured), and a bus-oriented design. At all Jcrctions tnrougnout 
tne system design tne simplest possible alternatives are inplemented consistent 
witn system requirements. Tnis simplicity results in low ovemead, a nig^iy 
structured system, and a mininun of special considerations. 



2.1 SYSTEM ORGANIZATION 

Tne total system is composed of four major components; tnree are in tne data 
patn, tne otner is tne system controller, as snown in figure 2-1. The ^♦OS data 
storage contains 0.5 x lO' bits of data plus 0.06 x lb' bits for tne error de- 
tection/correction code. Tbus, 12. 5X added capacity is required for tne error 
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correction fLTCtlon. The MNOS memory is organized into a wafer stack wnicn con- 
tains storage for tne code bits, spare functional memory, and tne data memory. 
Eacn wafer is divided into logically independent quadrants containing many 
4096-Dit memory arrays. A wafer is usable if it meets a predeteimined reouire- 
ment for tne total ntirber of operable memory arrays. No particular spatial 
distribution of usable arrays witnin a Quadrant is reouired, 

A system data buffer memory interfaces between tne MNOS memory and tne error 
detection/correction circuits. Tbis dual 64H»ord by 72-bit buffer memory 
functions as a data bit transposition unit. When data enters from tne error 
detection/correction circuits, it is stored 72 bits to a word (64 data bits 
plus 8 code bits) , until tne 64-4iord system data buffer is filled. The filling 
of an alternate data buffer memory is tnen begun. Meanwnile, tne first buffer 
memory is unloaded one bit position at a time, i.e., bit position 0 for all 64 
words, tnen bit position 1, tnen 2, etc. This bit transposition ensures tnat 
eacn bit of tne 72-bit encoded word is stored in a different memory array. TMs 
bit-slice arcnitecture ensures tnat snould any single memory array fail, tne 
lost data may be reconstructed by the error correction circuitry. 

The error detection/torrection circuitry appends an eight-bit code onto every 
64 bits of incoming data to produce a 72-bit word. (This word length has no 
relation to external word lengths since incoming data is serial and there are 
no word markers. ) The error detection/correction code has sufficient capability 
to correct any single error as well as to detect many nultiple errors. Gacn 
bit of this 72-oit wort is stored in a different 4k array. Thus, it is most 
likely tnat any memory failure error will be corrected and nence masked, as 
data are transferred out of tne system. The user will be unaware of the irv 
ternal failure; nis corrected data will be received at normal speed. A faulty 
memory array will be replaced at a later time by tne reconfiguration routine. 
A comprehensive description of this routine and the method of error detection/ 
correction coding is given in the report, "Error Control for a Mass Memory 
System," which is included as a separate volcm. 
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Tne system controller cooroinates tne timing oeteween tne various units in tne 
oata patn. It provides txjilt-in-test (BIT) capaoility as well as reconfigura* 
tion control for tne following; 

e For ^t4QS wafer components: replacing faulty memory arrays and con- 
troller circuits witn spares on tne wafer; also switching out entire faulty 
Quadrants and, wnere necessary i wafers. 

t For system data txiffer memories: switcning in spare memory cnips. 

f For error detection/correction circuits: switching in spare logic 
circuits. 

Tne switching in of spare units is facilitated by extensive use of tri-state 
Cxjses, power control » and BIT between oata file transfers. The system con- 
troller itself nas spare modules tnat may be switched in when malfunction is 
detected during BIT. 

It is thus se&i tnat tne redundancy built into tne system provides high relia- 
bility for extenoed missions. 

2.2 ^€MGRY WAFER ORGANIZATION 

An architecture is needed tnat can make maxinun use of functional memory arrays 
dispersed anong malfunctional arrays without using a disproportionate amount of 
silicon area for overhead firctions. Figure 2-2 presents an overview of such 
an architecture. A wafer is partitioned into four quadrants; each Quadrant 
possesses an independent operational capability. Within a Quadrant , memory 
arrays (MA*s) are arranged by rows, with each array in the row connected to a 
corrrnon row bus (X-bus) . There are no unioue lines to a memory array for chip 
select as is comnonly utilized with memories organized with discretely packaged 
units. Rather, address selection is accomplished over a 4-bit bus activated 
by an array controller (AC) located at the interior end of the row. Defective 
memory arrays are simply not accessed by tne AC. with most defects in a memory 
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Figure 2-2. wafer Laycxit 

array, no otner conseauences are assLned manifest. However, in tne less proo- 
aole cases wnere a defective array makes its neiyibors inaccessiDie or inoper- 
able, tne entire row is aoandoned by never accessing any memory array witnin 
tnat row and tnus never using tne AC controlling tnat row. Two different types 
of sucn faults may occur. In one type, an array would draw excessive power; in 
anotner, tne array would make tne x-txjs inaccessible. To rectify tne first 
fault, since tne power is switcned at tne AC, it can be turned off to tnat row 
wnere tne faulty MA resides, wnile power is still provided to otner rows. A 
similar operation is available for tne dus lines at tne AC. It can be noted 
from figure 2-2 tnat all tne bus lines for a row are snown connected to a trit 
of tnree AC'S. The entire x-bus is buffered at tnis point. Tnus, x-bus faults 
are not propogated tnrougxjut tne Quadrant but ratner are confined to a single 
row« 

Tne AC provides two main functions: one is gating between tne row and quad- 
rant, as described above; tne otner is routing data and control information 
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oetween tne cnannei controiicrs (CC's), -nicn arc located in tne first row on 
tre Quaarant, ana tne momory arrays. Tt^e routing function is effected in two 
separate steps. First in seouence, tne AC is addressed and locked onto tne 
Y-Dus wnicn carries inforrmtlon between a CC and AC'S. Oice an AC is locked 
onto a Dus, it responds to all corrrands on tne bus until unlocked. Tne second 
seouential step is to translate merory array nuitoers into actual memory array 
addresses, to present tne memory address on tne x-bus to lock tne addressed 
manory on tne X^s, and tnen to trarisfer information beti»een tne Y and x 
buses. 

Ea&> row is provided witn tnree AC's: one must be operational to make tne row 
accessible. Tt^t additional functional AC*s (up to two) are used as spares in 
tne event of failure of tne active AC. 

Tne Quadrant's first row contains several CC's; only one must be operational 
to make tne Quadrant usable. Hbwever, most Quadrants snould nave at least two 
operable CC's per Quadrant to ensure reliability. The major functions of a CC 
are to: 

• Recognize its quadrant address and lock onto tne Z-bus wnicn carries 
information to/from tne system controller outside tne wafer. 

• Translate data block nurbers into tneir AC addresses. 

• Buffer data and power lines between tne Z-bus and tne Y-txjs. 

• Route data and commands to/from tne AC. 

There are, tnerefore, two distinct levels of control in tne wafer arcnitecture. 
Tne CC interfaces witn tne off-wafer controls, responds to selected corrmands 
and routes commands and data, wnere intended, to tne AC. The AC interfaces 
witn tne CC, responds to selected cornnands and, in turn, routes commands and 
data to tne MA wnicn responds to corrrands and reads/writes data. 
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Tne Duik of a Quaarant is composed of a nurtoer of menxjry arrays (MA's). Tbe 
i^'s utilize nonvolatile HMDS memory transistors, wnicn allow oata retention 
witnout any standby power. Eacn MA can store A096 bits of data. Tnis data is 
accesed serially in groups of 64 bits, at a 2 K)lt/sec rate. Four MA's are 
active concurrently in a single Quadrant, i.e., 256 bits of data are accessed 
during any read/write cycle of a quadrant. 

The four MA's are accessed by first addressing tne CC of tne proper Quadrant, 
■me AC'S controlling two separate rows of MA's are tnen addressed, followed by 
a pair of MA's in eacn row. Once addressed, tncse four MA's act as a single 
unit witn tne CC and AC'S transparent to tne data flow. 

Two sucn quadrants are operated in parallel to allow the access of 64 8-bit 
words, or 512 bits, every read/write cycle of the memory. This (Xial quadrant 
configuratiwi is used to obtain the desired 5 rtjlt/sec system data rate. The 
nuioer of quadrants operating in parallel can be increased to give ni^er data 
rates. Sitrultaneous control of the dual quadrants 1$ perfotmed by the system 
controller. 



.2.3 SYSTEM CONTROLLER 



The off-wafer subsystem, including tne system controller is shown in fi^re 
2-3. The system controller is functionally and geometrically partitioned into 
three sections; eacn section has independent timing. They are: 



nswer kernel 

System microprocessor 

High speed sequencer. 



All sections are replicated and power switcnable to achieve specified reliabil- 
ity requirements. The power kernel swltcnes power to operational sections of 
the systOT microprocessor CRJ and to itself. This self-switching is accom- 
plished by majority voting at each of several redundant identical processors of 
the power kernel. Initially, power is supplied continuously to all operational 
power kernel processors. As units malfunction, they are powered down, and once 
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Figure 2-3. Off-wafer Subsystem 



powered down power can be turned on again only tnrougn remote control. Spare 
units are powered up as replacements, so tnat a full complement is operational 
until sucn time tnat all spares are exnausted. Tnis tecnnique ensures opera* 
tional power control as long as at least two power kernel processors out of tne 
original complement are operational and tne units fail one at a time. 

Operationally y a power kernel processor nas responsibility for comparing tne 
outputs of its replicates, cnecking the nealtn of tne syston microprocessor and 
power switching of itself and tne system microprocessor. The power kernel is a 
relatively simple single-cnip microprocessor. 

The system microprocessor controls the data operations for read, write, and 
erase; responds to system commnds; configures and reconfigures the system 
except for its own bootstrap kernel and the power kernel; carries out diag- 
nostics and BIT; and switches power to tne ni^speed seouencer, the memory 
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corrponents (wafers, Quaarants, CC's, AC'S, MA's), tne system data Duffer mencry 
and tne error detection/correction circuitry. 

Tne system microprocessor is a fairly sx'^isticated unit witn relatively nign 
speed capaoility. It rttist be contained in a few LSI circuits to make power 
switcning of its sections practical. Tne new 16-dit ni^ speed micrcprocessors 
neet tnese requirements. Reliability considerations will dictate tne level of 
replication needed to satisfy tne specified reouirement. 

The nign speed sequencer (HSS) provides tne detailed timing and sequencing to 
execute tne cotmBnos from tne system micrdprocessor. Altnougn tne HSS is 
relatively single, it is a very nign speed unit. It is composed of sequence 
control cnips, ROM's, and related cnips. For tne specified reliability, tne 
level of replication will be determined. 

By partitioning tne memory-system peripheral circuits to a relatively small 
replaceable unit level, nign reliability can be acnieved witn a mlninun of 
back-up nardware and witn no power loss to "not" standby circuits. Tne opera- 
tional cnaracteristic tnet allows tne use of "cold" standbys is "repair between 
data file transfers." Contoined witn real time single-data-error correction, 
tnis tecmioue provides tne desired system reliability witn reasonable expen- 
diture of power and weiy>t. 

2.4 DATA FUDW 

Inconing and outgoing data rate capability ranges from dc to 5 Kb/s bit serial 
rate as smmn in tne data flow cnart of figure 2-4. For a data file of 256 kb, 
read access is less tnan 1 ms; write access is approximately 7 ms (presuning 
an erase cycle before write). 

Data transfer out of tne system data buffer memory is at an 8-bit word rate of 
oc to 2 ND/s peak or oit rate to 16 Kt)/s. "me >40S memory accepts oata at tne 
san« oc to 2 Mb/s peak word rate for a 8-bit word, or 16 hti/t bit rate. 
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Figure 2-4. System Data Flow 

Tne 5 K«/s serial data rate at tne user interface becomes 5.63 Mb/s after tne 
8 error bits are appended to tne 64 data Pits. Tbe difference between tne 
capability (16 K3/s) and tne requirement (5.63 Mv/s) provides time for overnead 
functions. At tne reouired rate, approxirateiy 52 ms are reouired to load/ 
unioao a 4096 x 72-bit file. Additionally, approximately 7-<ns erase tine pre- 
cedes a data file write. Two 043S/S0S ping-ponged system data buffers are suf- 
ficient for tne oesiT^ed data rates. Thus, no supporting system component 
limits tne data transfer rate. 

2.5 WAFER POVCR MANAGEMENT 

To minimize power consurption, power switcning is implemented at various levels 
in tne wafer arcnitecture. As mentioned previously, tne major attribute of 
f^OS is its retention of data witnout standby power. It is, nowever, not suf- 
ficient to control power only at tne memory array. The controller circuits, 
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even tnougn tneir naioers are not as large as tne noroer of MA's, would consure 
an inorainate arrcunt of power witnout power mBnagerrent. 



For power consioerations, eacn of tne three circuit types, CC, AC, and MA, are 
divided into two sections: tne "address compare" with its associated circuitry 
and tne Daiarce of tne unit. In figure 2-5, the address conpare circuitry is 
srown as tne rectangle witnin tne respective unit interfacing witn tne corres- 
ponding nigper level bus. Of tne several CC's fabricated in tne Quadrant, only 
two operaole ones are connected to me power bus at wafer probe time to reduce 
power consurption. wnen power is supplied to tne quadrant, only tne two CC 
aooress compare circuits consone power. If one of tnese CC's snould oe acti- 
vated tnrougn tne Z-bus by tne system controller, tnat entire CC is powered up, 
wnicn causes power to be supplied via tne Y-bus to all AC address compare 
circuits in tnat Quadrant. 
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wnen one of tne AC'S is activated tnrougn the Y-bus, tne renBinaer of tne AC 
circuitry is powered up; tnis causes power to be supplied via tne x-bus to tne 
compare circuits of all tne MA's in tnat row. In turn, an addressed MA will 
nave power supplied to tne remaining portion of its circuitry, wnicn includes 
tne 409^it memory array itself. 

Tne net effect is tnat for ail tne MA's on tne wafer, only tnose active arrays 
and tneir respective controllers dissipate power. T^e visual image is tnen of 
a narrow patn of power flow tnat is first vertical tnen norlrontal. It remains 
for a few milliseconds; it is tnen cnsnged as access is required of anotner 
array. Tnis constantly cnanging, darting flow of power is distributed over tne 
entire wafer ana results in a close to uniform distribution of neat on tne 
wafer, dependent only upon tne distribution of usable components. 

Since, in tne case of writing data, an extended write signal is required, it is 
necessary to supply array power for tne entire write time, wnile data are being 
written into otner arrays in different Quadrants. Additionally, tnis capability 
must be apnieved witn a mininun amount of power consuned. To tnis end, a power 
bit flag is provided eacn CC, men set, tne power flag in tne CO provides power 
to only tnose MA*s tnat are set for tne extended write mode. Tne write mode 
is terminated for all MA"s in tne ouadrant by resetting tne power bit flag. in 
tne CC. . 

Power to eacn of tne wafers is controlled by off-wafer power swi tones. One of 
tnese power switcnes can switcn a group of 32 wafers. TMs switcning of power 
to wafers in groups* allows tne power consuned by tne CC associative decoders 
to be minimized. These offnvafer power switcnes, as well as tnose on-wafer, 
are controlled by tne addresses generated by tne system controller. 

2.6 SYSTEM OPERATION 

Immeaiately after, power is switcned to tne memory system, tne power kernel of 
tne system controller receives tne system enable signal and begins memory sys- 
tem initialization operation. Tne first step tne system takes is organizing 
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ine system controiier into a worKing configuration. Tnis is perforTT«d in tnree 
steps; first, tne power kernel organizes itself into a functional group of pro- 
cessors; tnen tne power kernel enadles an operational system microprocessor 
CPU; and finally tne system microprocessor CPU flnisnes configuring tne rest of 
tne system controller. System initialization is completed wnen tne system 
microprocessor nas configured tne error detection/correction circuitry, and 
fetcned and refresneo tne proper address tables. 

crce system initialization nas Deen completed, tne memory is ready to execute 
a systOT conrond. Tnis system conrrend is eitner for a data file operation, 
read, write, or erase, or tne periodic built-in-test. The periodic built-in- 
test scans tne memory wafers and removes any faulty components. Tnis test is 
used, only occasionally, to prevent nultlple failures from occurring in tne 
less frequently used areas of tne memory, e.g., tne spare memory arrays, to 
minimize tne prooabillty of losing data. 0«e tne periodic test is completed, 
data concerning any faulty components is sent to tne external system. 

The oata file operations are tne basic functions of tne system; tney manifxjlate 
tne data stored in tne MA's. The comond for tne file operations specifies 
tne type of operation, wnetner it is erase, write, or read, and tne oata file 
wnicn is to be operated. Tne system controller tnen fetcnes tne addresses 
used for tnat data file. Next, tne system controller initializes tne nign-speed 
seouencer to generate tne proper sequence of memory wafer corrmends. Tne nign- 
speed seouencer is tnen enabled and tne first pair of Quadrants is addressed. 
Snort ly after tne quardants are addressed, tne data transfer, if any, begins. 

During operations wnicn involve data transfer, tne nign speed sequencer also 
issues comrands to tne error detection/correction circuitry (EDAC). Tne EDAC 
encodes tne data being written into tne memory witn a single error-correcting/ 
aouDle error-detecting (SEC/DED) code, wnen data are being read out of tne 
menory, tne EDAC decodes tne SEC/DED coding bits and uses tnem to identify and 
correct all tne single bit errors in a group of sixty-four data bits. Snould 
an error be detected during a read, tne EDAC issues an error status message for 
tne systOT microprocessor wnicn indicates wnicn bit is faulty. Tnis information 
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is usea Dy tnc system microprocessor during reconfiguration, when an error 
status message is received by tne system microprocessor, it sets a status flag 
wnicn indicates tnat reconfiguration is necessary. This reconfiguration only 
occurs wr^n tne reconfiguration system conrand is received. 

Once a file operation nas been completed, tne memory is ready for anotner sys- 
tem corrond. TMs conrnand can eitner be anotner file operation or tne power 
down operation. However, if tne reconfiguration request status bit is set, tne 
reconfiguration commnd snould be issued, especially if anotner file operation 
is not to be executed. Tnc reconfiguration, corrmand causes tne system micro- 
processor to analyze tne error status data to determine tne component most 
likely to nave caused tne error. The questionable component is tnen issued a 
sequence of test canrands by tne nign speed sequencer. Tne system micropro- 
cessor monitors and analyzes tne responses of tne component for eacn test step 
until tne operability or non-operability of tne questionable component is 
estaDlisr«i. Once tne fault nas been verified, tne system microprocessor 
locates a spare and replaces tne faulty component. After all tne detected 
faults nave been removed, tne system microprocessor issues status information 
to tne external system wnicn indicates tnose components tnat nave failed. 

The final conimnd to tnc memory system is always tnc power down cormend. This 
command provides for an orderly power down for tne memory system. The system 
microprocessor is given time to store all of its addressing and status tables, 
and tnen power down tne memory wafers in tne proper seoucnce. &ice tne power 
down command nas been completed, power can be removed from tne system without 
any repercussions. 
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Section 3 
HWDWARE DESCRIPTION 

3,0 SYSTEM H/«OW<«£ 

Tne manoiy system arcnitecture consists of tnc wafer memory modules and tne 
off-wafer control circuitry. The main components of tne off-wafer circuitry 
are tne systOT controller and tne error detection/correction circuits. The 
wafer rrenory ncxXjles consist of tne channel controllers (CC's), the array con- 
trollers (AC'S) , tne memory arrays (Mft's) , and tne buses wnicn connect tnem. 

The off-wafer system controller supplies the intelligence of the system. It 
not only inteiprets tne system connBnds and supplies detailed instruction 
secxjencing to tne wafers, Dut also monitors the operation of all system compo- 
nents. Should a fault be detected, the system controller executes a built-in- 
test to isolate the fault, and then reconfigures the system to remove tne 
faulty unit. 

Tne off-wafer error detection/correction circuitry performs the encoding^decod- 
ing and error correction of the system data. In conjunction with tne system 
buffer menor/, tne error detection/correction circuitry under progranrod corw 
trol also performs a bit transformation of tne encoded data to ensure tnat eacn 
Dit of tne 72-oit word is stored in a different memory array. 

Tne Channel and array controllers implement tne memory aodressing by selec- 
tively Switching data, control information, and power to tne intended group of 
memory arrays. This addressing tecnnique minimizes the power consarption of 
tne memory wafers, which constitute the bulk of the memory system. 
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3.1 N€MDRY ARRAY 

The oasic corrponent of tne system is the 4k-oit MNC3S transistor memory array. 
T^e most sigiificant feature of tne memory array is its ability to retain data 
witnout any standdy power. A memory array is composed of 4096 mnCS transistor 
fwrory ceils, arranged in a 64 x 64 matrix, and hHOS transistor peripneral and 
control circuits. The control circuits consist of tnree main sections; tne 
data circuits, tne addressing circuits, and tne cnip instruction and timing 
control circuits (depicted in fi^re 3-1). 

The i-NQS transistor memory matrix occupies approximately 40 percent of the mem- 
ory array area, which measures approximately 17.3 mm^ (26,9 x 10^ mils^), Tbe 
NMOS data circuits associated with tne memory array occupy almost another 30 
percent of tne area; tne addressing, instruction decode, and timing control 
circuits make up tne remaining 30 percent. 




CLOCK 



Figure 3-1. Memory Array 
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CCL unlocK (CUL) causes all previously iocKeo CC's to unlock, inoepenaent of 
any CC afloresses. A CC wnicn nas oeen unlocked nas its active Dit clearefl, 
causing power to De snut off in botn tne CC itself ano tne associated y-ous. 
v^nen tnis Y-Ous loses power, tne rest of tne Quadrant, including all tne AC'S 
and MA'S, also loses power. However, tnere is a power dit in tne CC wnicn is 
set by a CC comand. wnen set, tnis power Dit maintains power to tne Y-ous, and 
all lower levels of tne quadrant, even after tne CC nas been unlocked. Tne CC 
power bit, in conjunction witn tne AC self-latcning power switcn and tne 
unclearable MA active bit, allows a quadrant to acnieve a minifflal power state 
wnere only tne CC decoders and tne MA's are fully powered. 

Tne purpose of tnis elaborate power switcning scnene is to canpensate for tne 
additional power used during tne extended write and erase operations, wen tne 
write voltage is applied to an m for 175 microseconds, or tne erase voltages 
for 2 tns, no additional data or control is needed in tne quadrant, so tne CC's, 
AC'S, AC decoders, and MA decoders cease to perform any useful function. Tnese 
exponents can tnen be forced into a low power state, tnrougn a conoination of 
CC ano AC unlocks, witn tne CC power bit set. Once tne write or erase opera- 
tion nas been caipleted, tne CC power bit is cleared and tne entire quadrant 
snuts down on tne ensuing CC unlock. Since a read operation ends after tne 
data is snifted out of tne quadrant, tne CC power bit is not set and tne quad- 
rant snuts down upon tne CC unlock. 

Tne nurtjer of CC decoders wnicn are powered at any one time is kept to a mini- 
mum by switcning power to groups of wafers. Tbis wafer group power switcning 
is performed by tne system controller during addressing, and only tnosc wafer 
groups wnicn contain tne current data file are powered. 

3.5 WAFER SUBSYSTEM 

Figure 3-12 illustrates tne system in its pnysical relatlonsnips, particularly 
witn respect to tne five itwwry module canisters. Bacn canister is separately 
connected to tne syston control I/O board. From tnis connector tne wafer stack 
bus runs tne lengtn of tne canister witn eacn wafer board soldered to it. 
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Figure 3-12. System Structure Block Diagram 

Tnerefore, its total lengtn Is just tnat of the canister Itself, Tbe system 
control is located on two txsards Just above tne I/O board and Inteicoonected 
via connectors on eitner end. Tne external system interface emanates from one 
of tnese connectors. 



Tne memory is composed of five memory module canisters containing 236 wafers 
eacn witn an average of 0.5 x 10^ bits of nonvolatile memory per wafer. To con- 
serve power and nvinlmlze peak power reouirements, eacn canister is suboivided 
into 8 independently controlled power groups sucn that only one power group out 
of tne total system is supplied power at any one time. From tnis restriction, 
it tnus follows tnat tne 18 data blocks «nicn comprise a single file, are 
located witnin tne 32 wafers of one particular power group out of tne total of 
W power groups. 
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Trie power control circuits arc locateo outside tne canister itself on tne PC 
Doara associated witn tne system control I/D. Two power control circuits are 
required: one for Vqj^ and one for Vqq* In eacn power control circuit are eignt 
separate power lines, only one is in electrical contact to any particular 
wafer, inoer system control, power is selectively applied to one bus at a time. 
Tnus, power is applieo to only 1/8 of one wafer stack (32 wafers) at a time. 

Operation is illustrated for tne case of writing a file wnerein six wafers are 
reouired to oe operational at any one time. Power is applied to tne power group 
wnere tne file of interest lies. TMs is represented in figure 3-13 witn power 
applieo to power group A and canister No. 5. Shown are six selected wafers 
wnere data is Deing written. Eacn wafer draws approximately 2 watts. Eacn of 
tne remaining 26 wafers of tnat power group draws only approximately 0.1 watt. 
(The associative decoders of tne 8 CC's are powered.) No power is crawn by tne 
wafers in tne remaining power groups of tnls canister. Approximately every 100 
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Figure 3-13. wafer write ft)wer Reouirement 
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Figure 3-14. R)«rer Kernel Organization 



by tneir NVM status iatcn, to power-up and initialize tne entire power kernel. 
If a majority of tne n-active microprocessors are functioning, an acknowledge 
signal is sent back to tne external system. The external system, being con- 
fident tnat a workable power kernel configuration nas been acnieved, pulses tne 
GO interrupt and tne power kernel, replaces all of its faulty microprocessors, 
and proceeds to start tne system bootstrap. 



Snould tne external system conclude tnat a functioning power kernel configura- 
tion nas not been acnieved, because no acknowledge was received or for any 
otner reason, tne external system continues automatic configuration by clearing 
tne system enable and setting tne second system enable. This second system 
enaole switcnes power to tne microprocessors designated as second active. This 
second level of mR configuration sends a first acknowledge, and tnen recon- 
figures tne entire power kernel so tnat tney are now designated as active. 
T^is new active hWR group sends a second acknowledge so tnat tne external sys- 
tem can switcn back to tne regular system enable. 
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Snoulo tne second ievei of Nt« configuration prove inoperative, tne external 
system can still attorpt to acnleve a workable hWR confi^ration oy dnjte 
force. This is accomplisned dy a set of master power control lines. Tnere is 
one of tnese raster lines for eacn power kernel microprocessor. w«n set, 
power can be directed to any single processor exclusive of its nvm status, or 
any otner local control. Similarly, power can be removed from any micropro- 
cessor by tnese lines. In tnis manner, one configuration is acnieved. Tnen 
eacn of tnese processors works to reconfigure tne power kernel until eacn 
microprocessor nas its proper NVM status, "me external system can tnen return 
configuration control to tne automatic mecnenisms of tne power kernel using tne 
system enable. 

Tne power kernel alters its configuration, under automatic control, similar to 
tne way tne external system controls tne configuration of tne entire power 
kernel. Once an operative nkF configuration is activated, tne good micropro- 
cessors will replace any of tne microprocessors determined to be faulty. The 
active processors can address and swltcn power to any of tne inactive proces- 
sors at tne various status levels. For eacn processor of an hWR configuration, 
tnere are otner spare processors at eacn of tne four status levels: active, 
second active, standby, and dead. To replace an active processor, tne second 
active processor is powered and tested. If operative, tne second active pro- 
cessor is switcned to active status wnile tne previously active faulty proces- 
sor is switcned to dead status. Oice a processor nas been switcned to dead 
status, it can only be revived by external system control of tne master power 
lines. 

Tne status of tne standby processors is tnen updated so botn an active and 
second active microprocessor is available to eacn processor component of tne 
NMfl configuration. Tne detailed control of tne status switcning is acconplisned 
by power enable and disable controls wnicn are addressed by tne operative hWR 
configuration. 

Tne power kernel is imajne to virtually all transient faults. The external 
outputs of eacn microprocessor of tne operative hWR configuration tolerate 
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transient faults oy Deing piped tnrougn a set of riardware majority voters. 
Transient faults internal to tne power kernel, sucn as test results and testing 
comparisons, are eliminated tnrougn software majority voting, and tne power and 
status control are all performed tnrougn hardware voting. Since delegation of a 
microprocessor to dead status is a drastic step, no microprocessor is assigned 
a dead status until it nas failed at least two test procedures. HDwever, tne 
master power control lines of tne external system can always revitalize tne 
microprocessors mistakenly assigned dead sUtus, should tne situation ever 
arise. 

Tne software voting procedure used by tne power kernel processors to configure 
tnoTtselves into a fault-free configuration proceeds as follows. EScn processor 
executes a test routine upon receiving tne GO interrupt. Tne result of tnis 
test routine is a ntirder wnicn reflects tne current operational capability of 
tne processor. This niiitoer is a form of cnecksun. Eacn processor receives tne 
test result from ail tne otner active processors and compares tnem. Any pro- 
cessor's test result wnicn differs from tne majority of tne otncrs is assorwd 
to be faulty. So after a repeat of tnis software test-ano-vote procedure, tne 
faulty processor is replaced. 

Cbrrnvrication between tne processors is accomplisncd tnrougn tneir universal 
voters, snown in figure 3-15. The status noitoer of eacn processor is sent to 
tne others by eacn processor loading a register in tne respective universal 
voters. The registers in eacn voter allow tnis transfer to be made slnultane- 
ously by all processors to eliminate differences in eacn processor's program. 
Tben each processor reads out tne status nuTtoers out of its voter. Qcn nattoer 
is Checked against tne others. After eacn comparison, tne processors output a 
power switch control word to tne universal voter of tne processor just checked. 
Should tnis processor's status nLntser not agree with at least one of tne other 
two, the Checking processor outputs a power-off control word. Otherwise, tne 
processor outputs a power-on control word. Tne voting network of tne universal 
voter then senos tne control word of tne majority to its power switch, wnicn 
then either srxits off or maintains power to its processor. Eacn processor, in 
turn, is power s*itcnea according to the majority decision. • 
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Snouid tne majority determine tnat a processor is fauity arxl so power it down, 
tne remaining processors power^jp a standby spare via this spare's LTiversai 
voter. Tne spare processor next to become active is cnosen according to tne 
status word stored in tne voter's NVM. Hbwever, snouid a reiiabiiity anaiysis 
snow tnat tnis dependence upon tne universal voter's power switcn and status 
word is too great, an alternate power switcning tec^nioue is being examined. 
In tnis alternate tecnnioue, a faulty processor is only powered down; tne 
replacement of tne failed processor is left to tne external system, ^^e pro- 
cessors would send out a signal to tne external system wnicn would cause a new 
processor to oe activated as a replacement via its master control line. 

Tne hm status register tnat controls tr*e power switcning is contained in tne 
universal voter cnip. Pomex switcning is accomplisned by reouesting tne second 
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stanatTy processor to de powered insteao of tne active processor. Tnen tne 
processor wnose hWM status is set to second standby is powered. Finaiiy tne 
status of all tne otner processor's nave tneir status upoated in sucn a way 
tnat one is active, one is second active, and at least one is spare (if enougn 
rerTBin)« Tne faulty processor is tnen relegated to a dead status. Thus, eacn 
processor of tne power kernel can be individually addressed by its hMR location 
and status. A similar tecnnioue is used for power switcning tnrougroot tne off- 
wafer subsystems. 

upon completing its self-test and reconfiguration routines, the now fully oper- 
able power kernel begins a test of tne system microprocessor CRJ. TMs is also 
accomplisr«d tnrougi tne universal voters, associated witn a system micropro- 
cessor CRJ. 

Tne majority of tne power kernel processors vote to issue an interrupt to the 
CPU under Question, to initiate tne CPU self-test. The result of tnis test is 
a CRJ status ncrrber wnicn is received by tne power kernel processors, via tne 
CRj's universal voter. Eacn processor tnen compares tnis CPU status nurder 
witn tne correct known value. Snould tne status nurters matcn, tne processor 
will send a pass signal to tne CRJ*s voter. iMhen tne rajority of tne power 
kernel processors pass tne system microprocessor, it receives anotner interrupt 
and begins a bootstrap of tne rest of tne system controller. If tne majority 
of tne processors fail tne CRJ, it is powered down and a replacement powered 
up, as was done for a faulty power kernel processor. 

Tne reliability requirements of tne power kernel demand tnat it be composed of 
only a few cnips witn snort and simple software routines. To accomplisn tnis, 
tne universal voter snould be developed as a custom LSI cnip. Tne processor 
will also be a single cnip microcomputer witn on-cnip ROM and R/W. using a 
small noTtoer of LSI cnips not only increases tne subsystem reliability, but 
also reduces its power consonption. To reduce tne power even furtner, OOS 
tecnnology will be used wnerever possible, and CMOS/SQS tecnnology will be 
used, if possible, to improve tne power kernel speed. RCA is developing an 
8-bit single-cnip microcomputer, 1804, wnicn uses CMOS/SOS tecnnology. Tnis 
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cofTOination of tne nlgp speeo, low power OOS^SOS tecnnoiogy, witn orucnip ROM 
and RPM, makes tne 180A especially desiraole. 

The software of tne power kernel is Dasically straightforward and sinple. By 
properly wiring tne corrponents of tne power kernel, all differences between tne 
routines in eacn processor can be eliminated. The key factor is making all 
status nuTtier comparisons tne same for each processor, because a processor does 
not know wretner it is voting on its own or another processor's status number, 
using tne sane routines in eacn processor simplifies tne testing and verifica- 
tion of the power kernel software. 

3.6.2 System Hicroorocessor 

The system microprocessor is a sophisticated nigrwspeed processor which per- 
forms tne main control functions of tne memory system. The system micropro- 
cessor functions are: 



e Responding to memory system commands, 

• lt)dating tne memory system status word, 

• Issuing commands to the ni^speed seouencers, 

• K)nitoring tne high-speed seouencer function status, 

• - Monitoring the error detection/correction status, 

• initiating and monitoring all built-in tests, 

• Rerfozming system fault diagnosis, and 

• controlling system reconfiguration. 



The system microprocessor configuration is shown in the block diagram of figure 
3-16. 

The system microprocessor is basically an interrupt-driven device. Noimally 
tne processor executes a monitor program which watcnes tne system interface and 
tne error detection/correction circuitry interface, as well as issuing connwds 
to tne high-speed seouencer. The other functions of the subsystem are executed 
in response to interrupts from tne other memory subsystems. An interrupt from 
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Figure 3-16. System Microprocessor 



tne power kernel causes tne system microprocesor CFU to perform a detailed 
self-diagxsing test wnicn infoxms tne power kernel if tne microprocessor CPU 
is properly fLixtioning. Interrupts from tne nigh-speed 'sequencer cause tne 
nriicroprocessor to perform tests of tne wafer controllers since an error nas 
occurred during memory wafer addressing. Interrupts from tne error detection/ 
correction subsystem (EDAC) indicate a data error and cause diagnostic analysis 
of tne status word to be performed to determine wnicn built-in test is neces- 
sary. The results of most built-in testing is interrupt-*)ased and causes some 
system reconfiguration. The software of tne system microprocessor is given in 
section 4. 



Tne diagnostic analysis and built-in testing done by tne system microprocessor 
rcQuire variable amounts of RAM as scratcn pad memory. To reduce power reouire- 
ments of tne processor subsystem, tne 4kb of scratcn pad is powered up in par- 
titions of 512 words, as needed. Since an interrupt-<Jriven system executes 
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mostly modular routines, tne 26kO of program R04 can be similarly partitioneo 
into 2k sections wnicn oecome powered wnen tne corresponding routine is exe- 
cuted. Tne wetTory control unit performs tnis power switcning of tne memory 
partitions as well as supplying tne merory control signals. 

Tne interrupt-based nature of tne system microprocessor reouires a multilevel 
prioritized interrupt structure. T^e interrupt controller supplies tne neces- 
sary decodirq and prioritization of tne interrupt signals from other subsystems 
as well as infonmtion specifying tne nature and origin of tne interrupt re- 
quested. Tne priority of an interrupt depends upon now critical tne situation 
requiring tne interrupt is. Error and. malfunction interrupts nave tne nigiest 
priority, because tney lead to otner situations wnicn may result in otner 
interrupts. For exat^e, since an addressing error interrupt eventually results 
in data error interrupts, addressing errors are nandled prior to data errors. 
Sucn an interrupt structure requires some intelligence in tne otner subsystems 
sirce tney must continue to carry out tneir basic functions, if possible, wnile 
tney wait for tneir interrupt to be acknowledged. 

Several coniiercial 16-bit microprocessors nave tne speed power products neces- 
sary for tne system microprocessor, as well as tne required sopnistication in 
instruction set and I/O interrupt capabilities. Tnree currently existing 16-bit 
microprocessors wnicn fit tnese requirements are: tne Faircnild 9440, tne 
Intel 8086, and tne Texas Instrunents 9900A. Because tnese processors nave only 
recently been developed, no final selection can be nade until tne development 
of all tnree is completed. A comparison of eacn of tnese is made in table 3-7. 

Because comtrication between tne system microprocessor and tne otner compo- 
nents of tne system controller, tne error detection and correction (EDAC), and 
tne TCmory wafers is critical, reliability may dictate tnat a reAjndant busing 
systen be used tnrougnout tne off-wafer subsystems. Altnougn tne level of 
redundancy will depend on tne results of a detailed reliability analysis, tne 
envisioned implerwntation of tne redundancy is to switch eacn bus as a unit, 
■me bus unit will consist of botn tne bus lines and tne bus transceivers at 
eacn end. This redundant configuration simplifies its implementation and 
reduces bus switcn complexity. 
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lAdLt 3-7. Mi'J^QWOLiSSOR ODNfARISGNS 



MICROPf^OCESSOM 


Tt9900A 


INTEL BOSS 


FAIRCHIL0 9440 


MAXIMUM CLOCK 


3 MHZ 


SMHZ 


12 MHZ 


AE6 TO REG AOO TIME 




1 S ua 

1,9 tm 




WTR - CPU 


O.SW 


O.SW 


LOW 


Pm-ER - SYSTEM 


<4.9W 


< %M 


< 5.3W 


REGISTERS 


ALL IN MEMORY 


• ON-CHIP 


• ON<HIP 


I/O CONTROLS 


MSI CUSTOM DESIGN 


MOSTLY ON<HIP 


•442 I/O CHIP 


INTERRUrrS 


> It VECTORED 


> If VECTORED 


> IS VECTORED 


INTERRUPT CONTROL 


MSI CUSTOM DESIGN 


MOSTLY ON^HIP 
PLUS t290A 


INTERRUPT CONTROL CHIP - 
MAY NOT BE DEVELOPED 


AOOITIONAL CHIPS 


NONE 


8224 CLOCK CHIP 


MAY REQUIRE 9441 MEMORY 
CONTROLLER 


TOTAL CHIPS 


« 30 


•20 


«20 


INSTRUCTION SET 


GOOD BUT SLOW 


80S0 CAPABILITY AND 
MORE INCLUDING 
MULTIPLY AND DIVIDE 


NOVA INSTRUCTION SET 


TECHNOLOGY 


|2L 


HMOS 


|3l 


TTL COMPATIBLE 


REQUIRES SPECIAL 
INTERFACE 


YES 


YU 


DEVELOPMENT STATUS 


IN PRODUCTION 


SAMPLES AVAILABLE 


STILL IN DEVELOPMENT 



Reliability also dictates tnat some amount of redundancy Oe useo In tne system 
microprocessor implementation. Tne system microprocessor will oe sectioned 
into a set of modules wnicn all nave unpowered redundant spares. The modules 
will consist of tne microprocessor CRJ, tne memory and memory control unit, 
ana tne I/O and interrupt controllers. All of tne modules, except tne micro- 
processor CRJ, will be tested and reconfigured under control of tne micropro- 
cessor CPU. Testing, and control of tne microprocessor CPU will oe governed oy 
tne power kernel. 

Once tne power kernel nas initialized a properly functioning microprocessor 
CRJ, cne CPU bootstraps tne rest, of tne system microprocessor so a fault-free 
configuration is activated. Tnen tne system microprocessor initiates and tests 
tne nign-speed seouencer (K6S) and tne error detection/correction circuitry to 
complete tne activation of an entirely fault-free off-wafer control subsystem. 
Tne system microprocessor tnen accepts a system command. The h6S and EDAC are 
set up to execute tnis corrmsnd, and tne memory system begins execution. 
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wniie tne memory system is executing a system canmnd, tne system microproces- 
sor is ronitoring tne progress of tne H6S and tne EDAC. Jould tne EDAC oetect 
any errors, tne system microprocessor will store the corresponding information 
aoout tne errors for use during built-in testing and reconfiguration. After 
tne system carmands nave been completed, the system microprocessor initiates 
any necessary built-in tests and use tne results df these tests to reconfigure 
tne system. The system microprocessor also continuously updates tne system 
status word to indicate both tne execution and error states of the entire 
memory system. 

3.6.3 HlQh-Soeed Sequencer 

The hig^-speed seouencer (HSS) is a subsystem of limited capabilities whose 
iTBjor function is to issue the specific instruction sequences and timing sig- 
nals to the wafers. Each Instruction stream is initiated by a cotnmnd from 
the system microprocessor. The instruction streams which tne HSS generates 
include not only the erase/wxlte/read seouences, but also many of the seoiences 
for testing and reconfiguration of the wafers. Since each of these instruction 
streare has many similarities, they are generated in blocks of subsequences to 
reduce duplication. The subsequences aerated to make up the entire instruc- 
tion strean are selected by flags set in the system microprocessor- «S status 
word by the system microprocessor, similar to conditional subroutines. Since 
tne HSS must issue different sets of instruction sequences as these flags are 
tested, tne hSS subsystem Is organized as a microcoded machine. The microcode 
controls tne data paths and components of the HSS as well as containing tne 
instructions issued to tne wafer. Besides issuing the instructions to the 
wafer, the HSS must also monitor the wafer's responses to the instruction 
stream, and issue control signals to tne EDAC circuitry to maintain some 
sytxnronlzatlon between the EDAC and wafer during data transfers. 

A sin^jlified block diagram, figure 3-17, snows the mein sections of tne ^6S. 
The microcode given in figure 3-18 is based on tne requirements of each of 
these sections. The central section conuins tne mlcrosequencer and the micro- 
storage. The mlcrosequencer generates tne addresses of the microcode. An 
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INTERFACE 



MtCROADORESS 
CONTROLLER/ 
SEQUENCER 



MICRGSTORAGE 
ROM 



EDAC 
INTERFACE 



WAFER 
INTERFACE 



-►EDAC 



WAFER 



Figure 3-17. Simplified hBS Block Diagram 
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Figure 3-18. Initial hSS Microcoae Format 
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Acvancea Micro Devices 29iO microseouencer is used nere since it is a single- 
cnip i2-oit microseouencer wnicn fulfills tne speed reouirements of tne H5S. 
Since so inucn is necessary to control tne instruction sequence sent to tne 
wafer, tne microseouencer operates at 8 NHt, allowing 4 microinstructions (3 
control microinstructions plus one microinstruction for tne wafer) to de 
executed in tne 500-ns cycle time df wafer instructions. 

To reduce tne power consurption of tne microinstruction storage ROM, CMOS ROM's 
are used. Because QCS ROt*$ do not neve tne cycle speed required, a pipelined 
configuration is used, as snown in figure 3*19, Four microinstructions arc read 
into tne pipeline registers siirultaneously. wnile tne next 4 microinstructions 
are being read, tne four pipeline registers are accessed by tne microseouencer. 

The system microprocessor supplies tne starting address of tne basic nrdcrocode 
routine for tne desired wafer instruction stream. Tne system microprocessor 
tnen selects wnicn of tne microcooe subroutines is to be included in tne basic 




Figure 3-19. h6S Diagram 
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routine Dy setting the proper flags of tne hss status register, as will be dis- 
cussed in section 4.8. These flags signify sucn tnings as read, write, or 
erase, and tne nurtoer of memory arrays involved* Tne HSS in turn updates tne 
corresponding flags in tne system microprocessor status register so tnat tne 
system microprocessor can monitor tne orogress of tne HSS and know tne current 
^€S state snould an error occur. 

Tne aooress of tne memoiy arrays for a given dau file are stored in a specific 
menory array on tne wafer. The system microprocessor supplies tne location of 
tne merrory array nolding these addresses in tne file address register. The HSS 
obtains tne addresses and stores tnem in tne address RAM buffer. Since eacn 
of these address words stored on tne wafer contains error code bits to maintain 
tneir integrity, tne EDAC nas its data paths arranged in sucn a way tnat the 
code bits can be checked and tne correct addresses supplied to tne HSS. The 
h5S sets up the data paths of the EOAC via the EDAC status register. Sxxild 
any fault situations occur in tne EDAC, the HSS is notified by flags set in 
the EDAC interrupt register. The K6S then notifies the system microprocessor 
of tne situation tnrouyi the system microprocessor interrupt register. 

Since power is only applied to a group of wafers as needed, the addressing in- 
formation loaded into the HSS address RAM buffer is also sent to the system 
microprocessor for wafer power group switching. The system microprocessor 
converts eacn memory Quadrant address into a power switch address via a looki^ 
in tne system microprocessor's processor power group table. The system micro- 
processor then switcns power to the appropriate power group. A data file will 
oe constrained to exist in only one power group to minimize tne power used. 

Many of the wafer instructions contain addressing information obtained from the 
address RAM buffer. The content of wafer instructions is dictated by the 
arrangement of tne multiplexer and latch contoination of the wafer interface. 
The arrangerent of tne wafer interface is dictated by the proper microcode 
secjuence. Tnis microcode seouence not only controls the content of tne wafer 
instruction, out also tne destination of this instruction,, namely, cnannel 1 
or cnannel 2. The destination is controlled by setting, the CHC (channel corv- 
trol) flip-flops. The CHC flip-flops are encoded to Indicate the four channel 
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access mooes, i.e., cnannei 1, cnannei 2, botn enamels, or no cnannei (aii 
cnanneis closed). 

Tne HSS is tnc only conponent of tne system wnicn can directly exert sucn 
exacting control over tne memory wafers. Therefore, its continued operation 
must be guaranteed. Replication is again used to obtain tne required relia- 
bii.ty. Hbre tne reliability scneme is to subdivide tne h6S into modules sirrv- 
ilar to tne main sections snown in figure 3-11. Eacn of tnesc modules will 
nave a nLittDer of unpowered spares, i^ncn tne system microprocessor causes tne 
HSS to execute a self-test, it monitors tne test results of this HSS execution. 
Sxxild tne system microprocessor tncn determine a module is faulty, it will 
switcn in a replacerrent. Tne exact norber of spares will be deterntined by 
reliability analysis. 

3.6.4 Alternate System Controller Con f iterations 

The rain driving forces benind tne design of tne system controller are nign 
speed, low power, and nign reliability. Eacn of tncse tradeoffs nas been 
balanced in tne current system controller design, wnicn consists of tne nign 
speed seouencer, tne system microprocessor, and tne power kernel. Alternative 
system configurations, wnicn perform tne same functions, will be described, 
witn tne impact of eacn on tne previously explained design goals. 

The first alternative system controller arcnitecture places more empnasis on 
tne niy speed seouencer, by incorporating tne functions of tnc system micro- 
processor in tne nign speed sequencer. To enable tnis modified nign-specd 
sequencer to complete tne tasks of botn tne nign speed sequencer and tne system 
microprocessor, it trust operate at a very nign speed, wnicn dictates tne use 
of bipolar te^nology. Siroe tne computational power of tne system micropro- 
cessor (Tcist also be incorporated, a 2900-type bit-sliced microprocessor is 
indicated. 

Tnis 2500-based nign speed sequencer must nave tnc same capabilities of an 
8066-type microprocessor: 6Skb address space mininun, 16-bit data words, and 
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16 prioritized vector interrupts. A 2900 system incoiporating tnese features, 
including a carry look anead generator to guarantee operation above 10 NHz (3 
to 4 tires tne speed of tne present nlgn speed sequencer), takes 13 major OIP's 
wnicn consone approxinBteiy 11. 2W. These nutters do not include memory or I/O 
circuitry, tncy only replace tne system microprocessor CPU and tr»e hign speed 
sequencer 2910 aodress controller. Tne circuitry replaced by this 2900 system 
only took 6 DIP'S and A.lW; tne 2900 system occupies more room and bums nticn 
more power. Botn tne system microprocessor and tne nlgn speed sequencer were 
originally allocated only 8 W total. 

Not only does tne increased nurrber of DIP'S in tne 2900 system increase tne 
size of tne active nign speed sequencer/system microprocessor, but also it de- 
creases tne reliability of tne system controller. Therefore, tne ntirtjer of 
spares rrust be increased to maintain tne reliability. Additionally, because 
tne complexity of tne increased size may result in a less testable system, tne 
diagx)stic cneckout of tne 2900 based system may take longer even tnougn it is 
operating at a mucn ni^r speed. 

The fact tnat a 2900 is microprogramned cannot be ignored. Eacn microprograrmed 
instruction takes a nortoer of microinstructions; the size of the program ROM 
may need to be increased. Every sucn increase in memory size affects tne ntirw 
ber of 2909/2911 microsequencers in tne 2900 system, which again impacts tne 
power, size, ano reliability. 

This 2900-based system controller can be further modified to incorporate tne 
functions' of tne power kernel. This can be accomplished by adding some 
MSI/SSIlogic to divide finctionally eacn 4-oit slice of tne 2900 system into a 
power kernel processor. This switch to tne power kernel is controlled by tne 
external 

system. Once operating as the power kernel, eacn 2900 slice would then perform 
a functional test and conpare its results with those of tne other slices and 
vote to remove any faulty slices. Once eacn slice was deemed functional, tne 
external system would tnen be signaled to switch tne 2900 slices back to a 
16-oit processor. 
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Tnis douOiy moaified system would reduce tne impact of tne larger size of tne 
2900 system; tne total nuroer of microprocessors would be reduced. This ver- 
sion of tne system controller also concentrates tne design proolems into one 
ratner complex microprocessor subsystem. 

Power would still remain a problem since most of tne 0.5 watts allowed for tne 
power kernel would be used by tne additional MSI/SSI logic needed to control 
tne individual 2900 slices. Also tne size would probably remain increased 
because tnis 2900 system must now be tne most reliable section of tne entire 
memory system. 

Anotner alternative system controller configuration is obtained by placing most 
of tne nign speed sequencer control functions in tne system microprocessor. 
Sucyi a system controller configuration would reduce tne nigh speed sequencer 
to not rrvicn more tnen a nign speed counter and memory. Bernaps a watt or more 
could be saved by sucn a simple nign speed seouencer. 

TTie additional burden of directly controlling sucn a nign speed sequencer, now- 
ever, may place too mucn of a processing load on tne current system micropro- 
cessor. In addition to monitoring tne nign speed sequencer's progress, tne 
syston microprocessor must monitor tne memory wafer responses. Since tne wafer 
responses to addressing occur in snort bursts, tne system microprocessor's time 
is fully occupied during addressing sequences. This occupation forces tne sys- 
tem microprocessor to accept data error status messages from tne EDAC subsystem 
only via direct memory access. Tnen, wnile tne memory wafers are snifting data, 
tne system microprocessor can sort and store any data error status messages in 
its data error file. 

During any single data block sequence, tne system microprocessor's time is now 
completely allocated to necessary functions, so subsequence selection, brancn- 
irq control in tne nigr speed sequences is no longer possible. Tnis loss of 
capability eliminates tne modular programming of tne nign speed sequencer, 
wnicn results in individual blocks of instruction for many sequence variations. 
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Aitnoug^ tne addlticy^I internal control steps of nign speed seouenccr programs 
are no longer necessary, tne program storage of tne nign speed seouencer is 
greatly iixreased. Therefore, no real component savings is acnieved by tnis 
TOdified system controller configuration. 

The final system controller variation to be discussed is the combination of the 
system microprocessor and the power kernel. Here, a nortoer of active system 
microprocessor CRJ's are firctioning together. Each of these CRj's performs 
evaluation tests on one another similar to the procedure of the power kernel. 
The two main differences of sucn a configuration are that the outputs of all 
active CPU's are voted upon and tne power consumed by this nultiple system 
microprocessor system is increased proportionally to tne nuroer of active 
CPU's. 

The reliaoility of this system multi-fflicroprocessor is greatly enhanced by this 
configuration, virtually all transient faults in the system microprocessor are 
reroved by the majority voted outputs. Also, a total failure of anything less 
than tne rajority of the active CRJ's will not crash the memory system or 
interrupt its operation during a memory system cornnand execution. However, 
tnis configuration forces tne system microprocessor to nave the greatest over- 
all reliability, and so results in an increase of not only the nurtoer of active 
CPU's, but also tne nurder of inactive system microprocessor spare subsystems. 

Tne power increase due to the increase of the nurtoer of active system micropro- 
cessor CPU's is nontrivial. For a TMR-toased system, the power increase is 8 
watts, for a total system power of approximately 47 watts for a write opera- 
tion. An NMR system increases similarly as N increases. 

Altnouy* each of tne system controller variations discussed here nave some 
aovantages, twy all nave significant impacts on tne reliability, power, and 
size of the systan. Additional variations of the system controller configura- 
tion are possible; any pernutation of power kernel, system microprocessor, and 
niywspeed seouencer can be nnade to fit tne speed/rellablllty reojlrements, 
and to perform TOst of the existing system controller function.. However, an 
additional cost of power and size results for each of them. 
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Many of tncse alternative configurations pusncs one or anotner of tne system 
controiier suosystems toward its limits. This tends to limit tne flexibility 
of tne system for future expansions of speed and memory capacity. 

3.7 ERROR DETECTION/CORRECTION AND SYSTEM DATA BUFFER ►€K3RY CIRCUITRY 

The error detection/correction and system data duffer memory circuitry perform 
tne encoding/decoding and formatting of data stored in tne memory system. The 
error detection/correction circuitry converts tne serial data to/from an error 
correction code word wnicn nelps maintain data integrity. (A complete descrip- 
tion of tnis tecnnioue is presented in Volane VI, Error Control for a Mass 
MBTory System.) The system data buffer memory circuitry converts tne encoded 
data to/from tne format used by tne memory wafers. Control for tne error 
detection/torrection and system buffer memory subsystem is provided mostly by 
tne nig%.speed sequencer. 

This subsystem will be implemented by a few custom LSI O40S/S0S chips. The 
OOS/SOS technology has tne needed speed capability and the desired low power 
consLTption. Existing or proposed LSI chips, sucn as Advanced Micro Devices 
2909 error coding cnip, could be used instead of tne custom OCS/SOS implemen- 
tation, but using tne custom implementation saves approximately 5 to 6 watts. 

3.7.1 The Code 

The simplest single error correcting/douole error detecting (SEC/DED) code, a 
linear cyclic block code is used. An SEC/DED cooe was cnosen because tne 
reconf igurable nature of the memory implies tnat the appearance of multiple bit 
errors would be very unlikely, and double error correction would thus lead to 
an unnecessarily complex implementation. However, a code whicn can also detect 
some of these multiple errors can readily be implemented without greatly in- 
creasing complexity. An SEC/DED code reouires a minimal Hamming distance of A. 
An SEC/DED code with 72-t)it words containing 64 data bits and 8 parity cneck 
bits has been selected. This is actually a shortened pseudo cyclic code based 
on 128-bit words where the extra bits are assoned to be zero. All of tne 
desirable characteristics of tne cyclic code are retained by this shortening. 
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A coded word is generated oy dividing tnat data wordi in tne form of a polynom- 
ial, by a generator polynomial wnicn defines tne code. Tbe generator polynomial 
used for tne code is g(x) 9 i ^ x * ^ For serial data, tne data word can 
oe encoded by using a linear feedback snift register as snown in figure 3-2D. 
Tbe eignt additional bits generated in tnis snift register are tnen appended 
onto tne 64 data bits to form tne 72-bit code word. 

^n a code word is retrieved » tne decoding process detects all one and two-bit 
errors, and specifically locates any single-bit error so it can be corrected. 
The decoding process is similar to tne encoding procedure. Ratner tnan gener- 
atirq the parity bits, it generates a syndrome wnicn identifies tne detectable, 
errors. If no errors occur, tne syndrome is identically zero. If a single 
error occurs, tne syndrome indicates wnicn bit is in error. After correcting 
tnis Dlt, tne resulting syndrome becomes ioentically zero, indicating no fur- 
tner errors. If tne syndrome does not become zero after correction, or never 
indicates a specific bit was in error, a detectaole rrtiltiple bit error nas 
occurred. Tnese multiple bit errors cannot be corrected by tne code alone, but 
may be resolved curing reconfiguration. 
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Figure 3-20. Linear Feedback 3nift Register Implementation for 
tne Pseudo-Cyclic Block Code Derived From 
g(x) = 1 ♦ X ♦ ♦ x8 
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3.7.2 Encoding Irolementatlon 

As data are snifted into tnc memory system, tnc data patn multiplexer directs 
tne data ootn Into tne 64-blt Information register and to tne linear feedback 
snlft register (LFSR), as srxawn in figure 3-21. During tne 64 snifts wnlcn fin 
tne informetion register, tne LFSR cycles eacn new data Dit into tne snlft reg- 
ister section via input taps. Eacn input tap corresponds to a term in tne gen- 
erator polynomlnal, 1 ♦ x ♦ ♦ x®, see figure 3-20. At eacn tap, tne new bit 
is exclusive-CF'ed witn tne preceding term to form tne input to tne next tap. 
After all 6A bits nave been snifted in, tne properly encoded 8 parity bits are 
in tne LFSR and ail 72 bits are sent in parallel to tne system buffer memory. 



In order to allow tne EDAC to better detect certain types of errors, eacn 72- 
bit word is designated as odd or even. Every even word nas a I's cover sequence 
modulo 2 added to it as tne data is snifted in. This cover sequence simplifies 
tne detection of nard stuck«at-one (stuck-at-zero) faults in a group of zeros 
(ones) . 




DATA MlTt/flCAD 



QATAffmOU 



Figure 3-21. Block Diagram; Pseudo-Cyclic Parallel Parity 
Decoding At 5 NfHz I/O Data Rate 
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3.7.3 Oecoainq Irolementatlon 



Since stored data are received by tne EDAC in parallel form, a sinpiified 
implementation can be acnieved by employing parallel parity decoding. In par- 
allel parity decoding eacn parity bit is generated by slnultaneously sumning 
tne data bits wnicn form tnat parity texm. In tne snortened cyclic cede used, 
tne same parity tree can generate eacn parity bit. Table 3-8 indicates tne 
bits wnicn are sarmea for eacn parity term and snows tnat eacn succeeding 
parity bit can be generated by rignt snifting tne data word one bit, witn zeros 
filling in at tne left. After only 8 snifts, tne entire syndrome nas been 
generated. 

As tne data are snifted out of tne LFSR and tne information register, tney are 
sent to one of tne two 8-Oit registers, snown in figure 3-21 as buffer I and 
buffer II. This pair of buffers allows continuous data transmission and 
oecooing after tne initial 8 snifts of tne first word. After an odd (even) word 

TABLE 3-8. PARITY CHECK MATRIX 
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nas nad ail but its last 8 data bits snifted out tnrougn buffer I (II), tne 
following even (odd) word is loaded into tne LFSR and infornBtion register. 
Ourlrq tne next 8 snifts, tne remaining data bits of tne odd (even) word are 
tranOTitted wnile tne even (odd) word nas its syndrome calculated and buffer 
II (I) loaded. As data leaves buffer II, tne all^ones cover sequence is 
removed. 

3.7.4 Error Correction Implementation 

Tne syndrome calculated by tnis particular parallel parity decoding metnod can- 
not be directly used to perform error correction on tne data in tne usual 
fasnion. The syndrome is first converted to a useful form by a specially en- 
cooed ROM look-up table. The calculated syndrome provides tne address to tne 
RGH; tr« resulting output corresponds to tne coIltti nurbert of tne syndrome, 
as indicated in table 3-8. The resulting output of tne ROM is loaded into a 
countdown counter. Should tnis counter reach zero during the 64 output snifts 
of tne current data word, tne particular bit being snifted out would be comple- 
mented; tne syndrome indicating it was in error. 

The same word loaded into tne counter is also sent to the microprocessor as an 
8-bit status word. The significance of each status word is shown in table 3-9. 
When a status word indicating an error occurs, an interrupt for tne system 
microprocessor is generated by tne EDAC which sets its error flag. The system 
microprocessor acknowledges the interrupt by resetting tne flag. 

Whenever a data error occurs, tne system microprocessor retrieves and stores 
both tne EDAC status word and tne data buffer address in an error buffer, as 
snown in figure 3-22. The EDAC status word indicates tne type of error and, 
for single-bit errors, indicates which MA is at fault. The data buffer address 
tells wnicn bit of tne MA word is bad, simplifying tne determination of hard 
or soft failures. The data buffer address also indicates if tne buffer itself 
nas become faulty. 
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TfldLE 3-9. BINARY STATUS WORDS rUR PSEUDO-CYCLIC CODE 
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As tliie permits, tne system microprocessor transfers the information from its 
data error Duffer to a data error file, wnicn is strxjcturcd as snown in figure 
3-23. Tne neader of tnis file is tne data file nurtoer, Tbe entries in tnis 
file are subdivided into MA nortoers. Under eacn MA nurtoer are tne data buffer 
address wr^re an error for tnat MA occurred, and tne nurtjer of errors tnat 
occurred at tnat word. Tbis error file structure allows tne compression of data 
eiTcr information to result in simplified diagnostics during built-irvtest 
(BIT). 

i^itiple bit error information is stored separately since it reouires more 
explicated diagxDStics. As nultiple bit errors occur, tne system micropro- 
cessor updates tne system status word to indicate an uncorrectaole error nas 
occured so tne outside world can react properly. The system microprocessor 
will maintain a count of multiple errors wnicn, witn otner information dictates 
tne priority of tne upcoming BIT. 
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BAD BIT NUMBER 



MULTIPLE BIT ERROR COUNTER 
MULTIPLE ERROR STATUS WORD 
MULTIPLE ERROR STATUS WfORO 



Figure 3-23. Error Status File Structure 
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3.7.5 System Data Buffer Memory 

During normal reac^wrlte operations tne system buffer acts as tne interface 
Detween tne EDAC and tne memory wafers. The system buffer is a pair of 72 x 
64 memory arrays wnicn perform a bit transformation on tne data passing 
tnrougn. During a write cycle, 72-bit words are seouentially loaded into one 
of tne buffer arrays from tne EDAC until its 64-word capacity is reacned. Tnen 
tne data are snifted out, in two 4-t)it groups, to tne memory arrays. To acnieve 
tnis tne buffer arrays are set up as an array of 4-bit registers arranged in 
rows of 18 and colLims of 64 as may be seen in figure 3-24. After data from 
tne EDAC is loaded into tne buffer array by rows, 72 bits at a time, tne full 
Duffer array sends data to tne memory wafers by colurms, 8 bits at a time. An 
entire colam of 8 bits is sent to tne mass memory before anotner column is 
started. 
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Figure 3-24. System Data Buffer Memory Array 
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Tr»is oit transposition of tne data allows eacn word to be distrioutea among 72 
memory arrays. Then, snould one of tne arrays be faulty, tne EDAC can correct 
tne resulting singie-dit error to preserve tne integrity of tne data. The 
reconfiguration and testing triggered by tne detected error minimizes any 
fTKjltiple-bit errors caused by more tnan one MA failure. 

To maintain tne reouired speed (5 Kft/s) of tne memory system, two Quadrants 
utilizing 8 MA*s are addressed sinultaneously. To facilitate tnis configura- 
tion, two colunns of tne buffer array are snifted out sinultaneously* Since 
tne EDAC cannot wait until tne entire buffer array is unloaded before refilling 
it witn new data, tne system buffer is arranged as a pair of buffer arrays, A 
and B. While recently filled buffer A is unloading, the EDAC is filling buffer 
B with new data, vwhen buffer B is full, txjffer A nas been emptied and tneir 
roles reverse. A signal is sent to the HSS, when a buffer array is filled, to 
maintain synchronization. 

For a read operation tne system buffer ftxictions in reverse. Colums of tne 
buffer array are filled with data from the memory wafer. Then row-by-row the 
re transposed data is sent to tne EDAC 

The system buffer is controlled by a set of three 7-bit counters which indicate 
tne row, colunn, and buffer array currently active. Two counters indicate the 
colLfm pair and row nurder for tne memory wafers, and the third counter indi- 
cates buffer array and row nait)er for the EDAC. These counters also provide 
the control response signals to the HSS. 

3.7.6 Additional EDAC Functions 

Besides performing tne normal read^write operations, the EDAC also performs tne 
encoding and decooing of Quaorant address data for tne HSS and of test oata for 
tne system microprocessor. The decoding and encoding of all data is acconw 
plisned in tne same way, except some data are supplied to tne hSS and some to 
tne system microprocessor. The HSS supplies mode control information to set 
up these three operations. 
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To read or write data in a certain oata file, tne addresses of tne associated 
trwncry arrays nust be fetcr«d from tne wafer. A minimal address fetcn time is 
acnieved Oy storing all eigpt 6A-b it address words in a single Quadrant, two 
words in eacn of tne 4 f^'s df tne first data oldck. TMs only requires two 
accesses, or approximately 120 microseconds. Since tnese address data are 
critical to tne nwory system operation, tney are encoded by tne EDAC to ^ar- 
antee its integrity. To prevent tne loss of address information caused by 
failures, copies of tne data are spread tnrougrout tne memory. i«r«n accessed, 
tne groups of four bits are loaded into tne system buffer in colum form as 
regular stored data, but loaded across a row in 4-blt groups until six 72-bit 
words are fonned. Eacn of tnese 72-bit words passes tnrougn tne EDAC for error 
cf«cking and is tf»n stdred in an address buffer in tne f6S. T^e mode controls 
from tne HSS alter tne control counter logic to facilitate tnis format cnange. 

Tne «S rode controls can also arrange tne system buffer to receive data from 
a sirgle MA. A 4-t>it snift register is used to convert tne 1-bit wide data 
stream into tne groups of 4-Oit worts reouired by the system buffer format. 
These 4-bit words are tnen loaded into tne buffer array as was done for tne 
address files. This added capability allows tne testing of a single MA. 

Sirre any malfLTXtion in tne EDAC results in some form of data loss, tne EDAC 
is tested before any data transfer can Uke place. The system microprocessor 
performs tne test of tne EDAC by supplying known data words in serial form to 
tne encoder ir^Juts. The EDAC tnen encodes tnese woitJs and stores tnem in tne 
systan buffer. When all erxrodlng nas been completed, the system microprocessor 
fetcT^es tne 72*it worts from tne system buffer in 8-bit groups across a row. 
Tne encoded wort is tnen checked by tne system microprocessor against known 
results. The decoding and error correction testing of tne EDAC is performed 
similarly, Dy loading coded test worts into tne system buffer in 8-bit groups 
across a row by the system mitroprocessor. The decoded serial data from tne 
EDAC is tnen cnecked, with tne resulting error states, by tne system micropro- 
cessor. The rode controls from tne «S also set up tne counter control logic 
for tnese operations. 



3-^ 



LEX1S497 



VOlUT« III 

Sirce tne EDAC, including tne system Duffer ana aooitionai control logic, con- 
sists of a few custom ODS/50S LSI circuits, tne system microprocessor i«juld 
permanently disconnect tne faulty cnip, as opposed to groups of cnips, and 
lock-in a good spare for a corrected EDAC configuration. 

3.8 POV£R ESTIMATES 

Tnroug^ size and integrated circuit complexity approximations, power estimates 
for tne various wafer components nave been calculated. Since eacn integrated 
circuit on tne wafer nas its decoder powered separately from tne rest of tne 
circuit wnen power is applied to tne circuit's bus, decoder power is calculated 
separately. Tne power for eacn type of decoder nas been estimated as follows: 

CC decoder 15 ntn 
AC decoder 7.5 nw 
MA decoder 7.0 itM 

Sir«e in tne maxinun case tner« are 11 MA's per X-bus, 77 mw must be allowed 
for eacn active x-Ous. During normal activities, power is applied to two X- 
Duses in a quadrant, wnicn results in a 140 mw total for tne MA decoders. In a 
Quaorant tnere are a maxliiun of 30 AC'S per Y-bus and only one Y-bus per Quad- 
rant. THUS, a 225 mw total for AC decoders in one quadrant, is Included. Tne 
total decoder overnead is tnerefore 380 mw per Quadrant. To reduce Quadrant 
overnead power durir^ write and erase sequences, tne AC and MA decoders nave 
been designed to switcn to an Inactive state tnat reduces tne power to 3.1 n« 
per decoder, or 155 mw per quadrant. 

wnen a wafer is powered, tne CC decoders are also powered. Since tnere are 8 
CC's on a wafer, 120 mw CC decoder power per wafer is used, wafers nave been 
grouped in sets of 32 wnicn are powered simjltaneously; 3.8 w per wafer group 
is tnerefore allowed for CC decoders. 
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Tnroug^ carparisons witn tne existing memory cnip, tne memory and controiier 
cnips nave oeen estinBted: 

CC 300 mw 
AC IX m 
MA 350 mw 

For an active quadrant tnere are four fully powered "MA's, t*o powered AC'S, and 
one powered CC for 1.9 watts total power per active Quadrant. During write and 
erase sequences tne controls assure an inactive state after addressing and 
require only 100 mw for a CC and 50 mw for an AC. An inactive quadrant tnen 
nas 1.6 watts total. 

The system controller subsections nave been estimated by totaling tne indivi<*- 
ual proposed LSIXSZ device power requirements. Ibe tnree subsystems require: 

System Microprocessor 5.0 W 
HigrwSpeed Sequencer 3.0 W 

Power Kernel 0.5 W 

System Controller 8.5 W 

Tne EDAC and system data buffer memory are implemented in custom CHOS/SOS 
LSI. l^eir power estimate is based on LSI OOS circuit information: 

ZDfiC k Buffer 1.0 w 

(If different for QOS/SOS, it will be lower.) Since tne EDAC and system 
Duffer are superfluous during erase sequences, tney are powered down during 
tnese operations. 

Tne total power switcnes for tne wafers and otner subsystems must also be irv 
eluded. The total of tnese nes been estimated: 

Power Switcnlng 2.0 w 
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Tne system peak power estinates are snown in taoie 3«iO for eacn seouence: 
reao, write, and erase. The mejor power variations between seouences are due 
to tne norber of Quadrants powered. Since writing ana erasing tne MA's requires 
more time, the sequences of different Quadrants are overlapped. Two quadrants 
are actively being addressed or loaded witn data, while four Quadrants are 
inactively applying an erase or write voltage to their MA*s. 

T«LE 3-10. WORST CASE PEAK PO)€R ESTIMATES (IN WAHS) 
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All power estimates for the single file operations, read^ write, and erase, 
are suimBrized in table 3-11. A single file operation begins with the 
fetching of 512 bit file address table; and concludes when the read, write, or 
erase operation is conpleted by the 64th word of the 18th Quadrant, even 
though the cornpletion may occur after tne last data bit leaves the error 
detection/correction subsystem. The times for each operation are included in 
taole 3-11. 

TABLE 3-11. POiCR FIGURES (EXauSIVE OF POWER SIPPLY) 
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Tnerc are tnree types of power values included in taDle 3-11: average power, 
peak power, and total power. Tbe peak power is tne worst case maxirrum power 
used during eacn of tne file operations. T^e5e peak power estinates nave been 
updated to include tne cnanges in tne mexintim narber of AC and MA decoders on 
tne X or Y busses. 

Tne average power reflects tne srall power cnanges wnlcn occur during tre ad- 
dressing of a guaorent and tne fluctuations in tne nuroer of active and inac« 
tive quadrants curing tne cycle of tne file opention. An assorption has been 
made tnat tne peak power cnanges at tne conclusion of tne instruction wnicn 
causes the power cnange. Tnis assurption leads to sligntly ( .iw) pessimistic 
power figures, but greatly simplifies tne calculations. 

The avenge power includes tne file address table fetcning wnicn uses about 
16. 3W for 202 microseconds. This fetcning is also included in tne total file 
operation times. The operation times given assure tnat eacn instruction is 
completed witnout any delay due to errors; tnat is, tne instructions are 
seouenced at tneir maxlnun rate. 

The total power reflects tne entire amount of energy used during a single file 
operation. This total energy is Just tne avenge power used during tne entire 
time of tne file operation (avenge power x time). 

Tne power estimates included in table 3-12 reflect tne input power to tne sys- 
tem power supply. A 70 percent efficient power supply is assoned; tnis 
explains tne A3 percent increase in all of tne power figures. 



TABLE 3-12. POVCR FIQURES (irCLUDIhC POUCR SlPPLY) 
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Section 4 
SYSTEM OPERATION 

4.0 N€MORY SYSTEM OPERATION 

The memory system acts as a secondary storage device for an external system 
mat supplies tne basic control signals and ccnimands to govern tne memory sys« 
tern operation. The first step of merrcry system operation is the initialization 
of tne system controller, whicn is oegun when the external system supplies tne 
proper control signals to the power kernel. After initialization is completed, 
tne external system supplies tne reouired system commands to the memory system. 
The memory system microprocessor receives and interprets the cortmands into 
instructions for the high-speed seouencer. The high-speed sequencer then gen- 
erates the proper sequence of memory wafer comnands. Before the higrwspeed 
seouencer can issue tnis sequence of memory wafer comrands, it rnjst be supplied 
the necessary addressing information by the system microprocessor. 

4.1 SYSTEM INTERFACE 

The system interface provides a connection between the memory system and the 
external systems. It is divided into tnree sections: tne data interface, the 
control interface, and the power interface. Eacn section, as snown in figure 
4-1, directly interfaces with a different subsystem. The data interface allows 
cormunication with tne error detection/correction circuitry, the control inter- 
face allows comnunication with tne system microprocessor, and the power inter- 
face allows cormunication with tne power kernel. 

4.1.1 System Power Interface 

The system power interface supplies power to the entire system via the power 
kernel and control signals to tne power kernel. RDwer is supplied from a single 
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Figure 4-1. System Interface 



2&-voit supply of cne system power supply. The memory system power sypply 
wnlcn Is assaned to Oe 70 percent efficient, provides power to tne NM3S/KN0S 
'wafers, plus tne TTL 5V. 



wnen tne power to tne system is initially selected on, only tne power kernel 
becomes active. The system enable signals control exactly tnose power kernel 
processors tnat are activated and start tneir test and initialization routines 
as described in section 3.6.1 and 5.1. W)en tne system enable signals activate 
an operable set of power kernel processors, tney respond witn an acknowledge 
signal. Sxxjld neitner of tne system enable si^tals evoke tne proper response, 
tne master control lines are used to effect a brute force reconfiguration of 
tne power kernel. 
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4.1.2 System Control Interface 



Most of tne comajnication between tne external system and the memory system is 
accompiisned tnrougn tne system control Interface. The system control inter- 
face is composed of the system status and the system command words. The 
system status word indicates tne current state of the memory system and the 
system connBnd word controls the memory system. 

4.1.2.1 System Status Word . The system status word is a 4-blt word which is 
monitored by tne external system. The status word not only indicates the mem- 
ory system states of busy, reaoy for a commndt or Just finishing a command, 
but also inoicates data errors and a need for reconfiguration. The significance 
of each acceptable status word is snown in table 4«1. The 4 bits whicn comprise 
tne system status word are known as: busy/ready, completion, reconfiguration, 
and bad file. 

The busy/reaoy bit and the completion bit indicate execution status. The 
busy/ ready bit, bit Q, indicates whether the system is busy currently executing 
a system corrmand or some self-initiated comBnd, or it is ready to receive a 
new system conrond. The completion bit indicates that the memory system has 
Just finished tne execution of some process. The completion bit is needed, in 
addition to tne busy/ready bit, to indicate, among other things, that tne sys- 
tem is ready to power down. Qenerally the completion bit and tne busy bit can 
never be set together, these are the only two mutually exclusive status bits. 
Sxuld both the busy and completion bits be set, an invalid status word or a 
system microprocessor error is indicated and appropriate external action should 
be taken. 

The reconfiguration bit ano tne oao file bit indicate error status. The recorw 
figuration bit, when set alone, indicates that either only single-bit errors 
have been detected in tne data file Just read, or that previously detected er- 
rors which nave not been removed by reconfiguration remain. Qenerally the 
reconfiguration bit is set only at tne end of a file operation, usually a 
read, with tne con^ietion bit. The bad file bit is inmedlately set when an 
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TABLE 4^1. FOUR-aiT SYSTEM STATUS HORD 

Dit 0 Dusy/reaay 
Dit 1 corrpletlon 
Dit 2 reconfiguration 
bit 3 bad file 



S tatus word Status Bit Indication 
ready - no errors 
busy - no errors 
corrand conrpieted - no errors 



0000 
OOQl 
0010 
OQll 
0100 



0101 

Olio 

0111 
1000 

1001, 

1010 

1011 
1100 

1101 

1110 

1111 



ready - need reconfiguration 



busy • reconfiguring 



conrand conpleted • need 
reconfiguration 

ready « bad file 



busy « bad file 



corrmand completed - bad file 



ready - severe failure 



busy - severe failure 



CGrnmnd corrpleted * severe 
failure resolved 



Meaning 

awaiting system corrmand 
executing system conmand 
awaiting system cormBnd 
invalid status 

either single-bit errors were 
detected in current file or 
errors still remain from prev« 
ious file - awaiting system 
conmand 

executing reconfiguration and 
test routines on previously 
detected errors 

detected errors still remain - 
awaiting system conrand 

invalid status 

multiple bit errors were detec- 
ted in current file - awaiting 
system conmand 

rrultiple bit errors detected in 
file currently being operated 
on 

detected nultiple-bit errors 
still remain - awaiting system 
coniTBnd 
invalid status 

system microprocessor was urv 
able to resolve severe failure 
- awaiting system command 
executing reconfiguration and 
test routines on detected 
severe failure 

successfully recovered from 
severe failure - awaiting sys- 
tem conrand 
invalid status 
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uncorrectaDle multiple-dit error is detected, it indicates tnat bad data is 
present in tne current data file, ?ould tne dad file bit be set, tne recon- 
figuration bit is not also set at tne end of the operation, since a multiple- 
bit error also indicates the need for reconfiguration, Botn bits may be set, 
However, snould the situation arise wnicn reouires tne memory system to stop 
executing its command and test and reconfigure some critical subsystem. Sucn 
a situation can only be caused by a severe failure. Oily a detected malfunc- 
tion in the high-speed sequencer or the error detection/correction circuitry is 
classified as a severe failure. 

All four bits are evaluated together to ascertain the current situation prop- 
erly. For example, when tne error status bits indicate a severe error, bit 0 
snowing busy indicates that tne system is executing test and reconfiguration 
routines to remove tne fault. A completion bit indicates the fault has been 
removed and tne system command should be reissued. When bit 0 snows the system 
ready, the fault could not be resolved and system initialization should take 
place. 

A. 1.2.2 System CormBnd Vtord . The six system corrmands are organized into 
sixteen bit cornnand words, which are given in figure A-2. The first three bits 
of the cormand word foxm the operation codes which differentiate the commands. 
The next twelve bits specify tne data file to be used in tne single-file 
operations. The 16th bit is a parity check bit which provides some error 
detection in tne system command interface. 9vxjld the parity bit indicate an 
error, tne parity bit check line is made nign to indicate the cormand should 
be retransmitted. 

The single data file operations are the operations basic to the memory system: 
erase, write, and read. Each of these commands must supply the 12-bit data 
file aoaress of the file to be operated on. The data files are all 2564<bits 
(plus 32-kbits of coding), and tnere are a total of 2048 files. 
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COMMAND 

NONE 
READ 
WRITE 
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RECONFIGURE 
PERIODIC BIT 
POWER OFF 
NONE 
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Figure A-2. System Conronds 



The multiple data file operations are more concerned aitn the menery system as 
a wnole: execute reconfiguration, execute system-wide periodic txjilt-in-test, 
ana prepare to power down. The reconfiguration connand is generally performed 
after all file read operations nave been completed. Ihe reconfiguration cotmand 
tnen causes tne memory system to test and reconfigure tne memory wafers, to 
remove tne faults detected by tne error detection/correction circuitry. Since 
specific infonrati'on about tnese faults nas been stored away, this testing and 
reconfiguration is ratner brief and efficient. In contrast, tne system-wide 
periodic bull t-irv-t est covers tne entire memory system, wnicn includes all tne 
spare memory arrays and controllers. Because of tnis, it is time comsiining and 
snould only be executed when necessary. Periodic execution of tne system-wide 
test is still necessary botn to maintain tne efficiency of regular reconfigura- 
tion, and to prevent a build-up of faults tnrougnout tne system. 
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upon completion of eitner tne reconfiguration or periodic buiit-in-test pro- 
cedures tnc memory system outputs data concerning tne amount and type of units 
wnicn were replaced. This data specifies only tne largest unit wnicn nas been 
replaced, since » for example, wnen a row of MA*s is replaced, only the fact 
tnat a row nas been replaced can be determined, and not tne noroer of MA*s in 
that row. A log of all this information for the life of the memory is stored 
i 3 special file. The entire reconfiguration log is read out by tne memory 
when the reconfiguration conrend is received without the reconfiguration status 
bit being set. 

The prepare for power down command allows tne system to execute an orderly 
power down, as reouired. Upon receiving the power down conmand, the system 
executes a finalization routine which, at completion, outputs an error-free 
status word. Once this status word is set, power can be removed from tne sys* 
tern with no adverse effects. The all-zeros and all^ones op codes are not 
assigned any carrmend to help prevent spurious execution during system power 
transitions. 

A. 1.3 System Data Interface 

The system data interface provides the data link between tne external system 
and the memory system. In addition to the bidirectional data line, a pair of 
control lines are also provided in tne interface. These data control lines al« 
low proper syncnronization of the data transmissions between the systems. Tne 
first control line, send/receive, gives notice to the external system that the 
data transmission is about to begin. The 5-M1z clock line begins cycling two 
clock cycles later, and data should be moved one bit for every clock cycle. 

4.2 ADDRESSING STRUCTURE 

All addresses presented to the memory system refer to a data file and nave no 
Identity with a particular pnysical location. Tnese are called "data file num- 
bers.** As' snown in figure 4-3, a data file norber is 15 bits plus one parity 
bit wnere eacn 12 -bit address refers to a data file of 256 kbits, and tne 3-bit 
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Figure 4-3. Memory Addressing Structure 



operation code specifies tne memory operation performed on that file. T^ree 
levels of translation are performed on tne data file minder before it arrives 
at a pnysical address «nicn references memory arrays. Eacn translation occurs 
at an organizational feature of tne memory structure. 

Tne 12^it oata file nunoer references a taole «nicn translates it into a set 
of 18 pointers, eacn referencing a block of 4 MA* s in a particular Quadrant. 
Eacn pointer consists of a 13-bit quadrant address and a data block norber; 
togetner tney form tne address infoxmation presented to tne wafer. T^e OLjadrant 
adoress selects one quadrant out of 8192. me 5^it data block ncrrber is tnen 
presented to tne selected ouadt^nt. TMs serves as an indirect address for two 
pairs of addresses wnicn identify two different rows in tne quadrant; eacn con- 
tains two 4G96-oit memory arrays. Eacn row nas a A^it address. *me data block 
nijitoer also references a 3-bit array pair niirtoer wnicn provides an indirect 
address for tne lookup table located in tne array controllers. TTie result is 
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two 4-oit aMTcsses wnicn give tne pnysicai locations of tne two referenced 
memory arrays In tnis row. witn tne similar two arrays in tne otner referenced 
row, 16 kbits nave tnus been referenced in one duadrant from tne data file 
address. Similar reference to 17 otner Quadrants produces a file of 256 kbits 
of data witn tneir 32c error detection/correction bits. A map of tne morory 
appears in figure 4^. 

Translation of tne file norter to tne addresses of 18 data blocks comprising 
tne data file is accomplished in a four-step sequence as illustrated in figure 
A«5. This translation provides tne mecnanism for a fast and efficient program- 
controlled access of tne reouired data, as well as several desirable cnaracter- 
istics. Most irrportantly, tne file data blocks are completely relocatable, 
independent of eacn otner and of tne file nanders. Quadrant addresses are conw 
pletely independent of particular canisters (tne wafers are noused in envirorw 
mentally independent canisters) and tneir power groups (8 power groups to a 
canister) • Tne f our--step sequence follows: 



1. Interpret address of file address table from specific bits in file 
nuTtDer. 

2. Look up power group in wnicn file address table lies. 

3. Access file address table. 

A.- Access data blocks in sequence using file address table and power 
group table as required. 

Upon request for a file, tne file norther provides tne virtual address for tne 
file address table, wni^i is translated by tne ouadrant-to-power-group table 
(QPGT) to a particular physical area in memory. From there tne file address 
table is fetched and stored in active RAM for use as tne various data blocks 
are required. As tne entries of tne file address table are used to access the 
various data blocks, tne QPGT is used to obtain tne physical address for wafer 
power switching. 
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Figure 4-4. Memory Map 
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Figure 4*5. File Miroer to Relocatadie File Data 
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The file adoress taole of tne aaca olocks associated witn eacn file is recorded 
at tne time of configuration in a reserved (relocatable) areji of tne mass menw 
ory. Tnis file address table is configured as snoiin in figure 4^ witn 18 
entries per table referencing tne 18 data blocks tnat compose a data file. 
Thirty-two file address tables are arrayed per block of A, 4k-bit MA's (figure 
A.6b). The 18 entries of tne file address table are packed eight 64^it HA 
words with error detection/correction bits included in eacn word. The table 
itself is duplicated in another Quadrant in a different canister for backup. 
Two words are unused in the 8-word file address table, witn the otner six words 
containing tnree entries each. Eacn 18-bit entry in the file address table is 
COTposed of a 13-bit quadrant nurber and a 5-*)it block nurtoer sufficient to 
identify tne desired data within tne mass memory (figure 4-6c). 



To locate the file address table associated witn a particular file norder, tne 
file nuTter is interpreted in its 12-bit format into a 7-bit Quadrant aodress, 
and a 5-bit table nefrder from most to least significant bit position respec- 
tively, as srxjwn in figure A>7. Since the Quadrant address is a virtual address 
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FILE ADDR TABLES HAVE PREASSIGNED LOCATIONS (DATA BLOCK #1) ON A WAFER 



(assignable under program control and subject to change during reconfiguration) 
its particular real physical area location nrttst be obtained from tne ouadrant* 
to-power-gro(jp table described in tne following section. Tne seouence to 



a. using tne 7 most significant bits of tne file norter (tne Quadrant 
nutter) look up tne corresponding canister nutter and power group nutter in tne 

QPGT. 

b. Activate tne indicated power group* i.e., switcn power to tnis area, 
and access tne Quadrant indicated. 

c. Access in block #1 of tnat Quadrant, tne table indicated using tne 
five least siT^ificant bits of tne file nutter. 



81162 



Figure 4^7. File Fitter to File Address Table Location 



access a requested file is as follows: 
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d. Load tne file address table tnus found into Rm and access tnc 18 oata 
olocks listed in oraer using tne QPGT as required. 



Tne QPGT supplies tne present pnysical area location of the relocatable ouad^ 
rant address. As snown in figure 4-8, tne table nas approximately 5,000 entries 
of 6 bits eacn, wnere eacn entry corresponds to a particular ouadrant address 
translated to a 3-bit canister ntnt)er and a 3«^it power group witnin tnat can^ 
ister. Tne table as stored in RAM is in 8«t)it bytes per entry witn parity 
cneck used eacn access time. The table is replicated in preassigned nonvola- 
tile bulk memory to. tne degree to be detexmined by a reliability analysis. 

The Quadrant«to-power-group table is updated any time reconfiguration moves a 
Quadrant to a different power group. Of course, any update must be made to all 
replications of the table. The table is restored in RAH from bulk memory dur- 
ing reinitialization after power tumoff or power failure. 



A. 2.1 Quadrant-to^ower^roup Table 
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Figure A-S. Quadrant-To-Rower-Group Table (QPGT) 
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Figure 4-9 snows tne organization of tne array controller map (AO*) located in 
tne CC. Tbe !5-t5it data block nurtoer references one of 32 word pairs. Eacn word 
contains 7 Dits: 4 give trie AC row address and the otner 3 provide tne array 
pair noTtoer in tnat row. Associated witn tne AO^ is a snail (16-word x 2-oit) 
merory wnicn gives tne active AC address, indicating wnicn of tne 3 candidate 
AC'S is tne presently-active one. 

The ACM stores not only tne actively assigned row and array pair nLirtiers, but 
also any spare aodresses of tne Quadrant. These are differentiated oy storing 
an all-zero double word oetween tne locations containing tne active and spare 
aooresses. The last two locations of tne KQ^ contain a special code woro 
wnicn is used during tne Duilt-in-tests of tne CC. 
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Figure 4-9. Array Controller Map 
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Also associated witn tne array controller map is the spare row counter (SRC). 
The SRC is used during reconfiguration to read and write data of tne ACM, and 
to transnit data between tne AO and tne system controller. Tne system coo- 
troiier reads an AOI location oy loading it into tne SRC and tnen counting tne 
niitoer of increments tne SRC takes to reacn zero. 

Tne array controller map allows for tne storage of 512 kbits per Quadrant, 
i.e., 32 entries x 2 array pairs/entry x 2 arrays/array pair x 4 koits/array. 
Ttx)ugn current yield figures indicate tnat only approxlnetely 128k bits per 
Quadrant can be expected, tnls size of AO^ allows tne storage of information 
useful during reconfiguration, and provides for tne greater Quadrant capacity 
obtainable witn tne 4 micrometer, 8 kbit MA witnout redesign. 

4.2.3 Array Map 

Figure 4-10 presents tne array map (AM) located in tne AC. It contains 8 paired 
entries. E6cn paired entry is referenced by tne 3-bit array pair nurter. The 
5-t)it entry is tne pnysical address of tne referenced array. 

As in tne array controller map, tne AM stores botn agtive and spare MA ad- 
dresses separated by an all-zero wort pair. T^e last two locations also store 
code words used during array controller testing. Hie spare array counter (SAC) 
perfoxms tne same function for tne array map tnat tne SRC did for tne ACW. 

4.2.4 Address Assionment 

Figure 4-11 illustrates tne tecnnioue of address assignment to tne memory ar- 
rays in a row witn tneir subseouent address entry into tne array map. In tne 
top row, tne first array nearest tne AC is address "1" witn arrays furtner from 
tne AC naving increasing address values. A maximum of 16 array addresses are 
available; most rows will nave considerably fewer MA's. If 16 arrays in tnis 
row are found to oe operable and 14 assigned active status, 2 will remain for 
sparing. The arrays assigned to active status are those witn addresses of 
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lower nLinerlcai value, ine next address in seouence wnlcn was not assigned is 
stored after tne all-zero word pair of tne array map and loaded into tne spare 
array counter (SAC), When reconfiguration occurs, it will be the first to be 
used as a spare. 

The second row in figure 4«*11 presents an exarrple in which 8 operable arrays 
were found from 16 candidates and 6 were assigned active status; therefore, 2 
are in reserve. The array map is built from the operable array information. 
In this exarple it is assancd that 3 array pairs were required of this row. 
Thus, the 0-tn entry pair was assigned ntirbers 1, 2; then 3, A and 7, 10, with 
11 and 12 entered into word 4 of tne array map; word 3 is stored with all zeros 
to separate the active and spare addresses. (Assune the missing nurbers in the 
secxjence of assignment correspond to faulty arrays.) 

It is now assumed that built-in-test (BIT) has determined that array 10 in this 
row has failed and rnjst be replaced. The reconfiguration routine first loads 
the first half of word 4 into the SAC, after finding it follows the all-zero 
words, and enters it (11) into the entry where 10 was previously. The sudsti- 
tuted array whose address is 11 may or may not be operable. The reconfi^ration 
routine determines its operability. If it is operable, reconfiguration is com- 
plete; if it is inoperable, the SAC is loaded with the other spare addresses 
until the first operable array is addressed. If no operable arrays remain in 
this row, the data are reassigned to another row. 

Because operaole arrays are mapped on tne wafer, there is a minimum amount of 
off-wafer overhead. The only temporary external storage reouired is one 18-bit 
word per each 16 kbit block of data (18 words total) in the active data file 
and the QPGT. 

4.3 ^€HORY OPEPATION 

Data are written into or read from memory after the proper selection of CC, 
AC'S, and MA*s to form a channel. Four memory arrays, in a single quadrant, 
are accessed in parallel by each channel. To achieve parallel access of eight 
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MA'S total, four data lines, eitner data bus E or data bus F, are associated 
witn eacn cnermel as was snown in figure 3-10. TTxDse data buses are tnen sub- 
divided into two pairs of data lines, data bus C and data bus 0; eacn of tnese 
data line pairs are tnen split into tne single data lines, data bus A and data 
bus B, associated witn tne individual HA*s. 



4,3.1 Read 



To perform a read file operation, a pair of cnannels are opened by addressing 
two Quadrants (CO, two rows (AC) in eacn quadrant, and two memory arrays in 
eacn row. A typical setup seouence, to open a single cnannel for a read is: 



Open first cnannel 
Delay 

Select first row 
Delay 

Select first array 
Select second array 
Exclusively select second row 
Delay 

Select tnird array 
Select fourtn array 
Reselect first row 
Delay 



Tne delays in tnis seouence provide sufficient time for eacn cnip to acnieve 
full power. Hiring eacn of tnese delays, similar cormends are sent to anotner 
Quadrant to open tne second cnannel. Tne cnannel control signals are used to 
indicate wnicn cnannel is to execute a specific conrand. At tne end of tne 
final delay, all eignt MA*s in tne two cnannels nave been addressed and 
assigned a specific data line. 



upon completion of tne addressing of tne eight memory arrays, tne data is 
transferred to tne system data buffer by tne following seouence: 



Load word address 
Read commnd 
Transfer data 
^ift out data 
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Eacn of tnese carmBnds are executea Dy tne eignt memory arrays in tne two 
cnannels in parallel. Tne snift ccmnand is followed by sixty-four clock 
cycles wnicn completes tne transfer of data over tne eignt data lines to tne 
system data duffer memory-. Tnis entire sequence, as snown in tne upper portion 
of figure 4-12, requires approxirotely 90 microseconds to execute. 



inis 90 microsecond seouence is repeated nine times to access a total of 72 
MA'S in 18 ouaorants. (The two aoditional cjuadrants, eignt MA's, contain tne 
error detection/correction Pits.) After tne 18 Quadrants nave oeen accessed, 
sixty-four 72-Oit words nave been assembled in tne system oata Puffer. Tne 
system data Duffer is filled in tnis manner a total of 64 times, after wnicn 
tne entire 256k data Pits, as well as tne 32k error detection/correction bits 
of a oaca file nave oeen reao out. Tnis read procedure is consistent witn tne 
one-Pit correciaPle requirement Py naving placed eacn bit of tne 72-Pit encoded 
word in a different MA. Tnus, witn tne failure of any single MA, tne oata is 
fully recoveraole. 
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4.3,2 write 



The MA recxjires a block erase preceding data writing. The erase, as snown in 
tne center of figure 4-12, is a 2^s period following tne proper setup (similar 
to reao setup witn tne last comrBnd "erase" ratner tnan "read"). The erase 
canmand can oe issued to as many MA's in parallel as tne power budget allows. 
For exarple, if 0.35 watt is reouired per MA and 2.8 watts are allottea for 
on-^afer dissipation, then a data file of 256 kbits would be erased in tnree 
2.13 millisecond periods as snown in figure A.13. Following erase, data are 
written into eignt MA's in two Quadrants at a time preceded by the proper set- 
up- The write seouence into an array, as it appears in tne bottom of figure 
4-12, begins mjcn like the read, witn a 90 microsecond period when tne eignt 
NW's are accessed sinultaneously via tne two channels. Following tne snift in, 
rcwever, a write period of approximately 180 microseconds is reouired, as shown 
in figure 4-14. Tne setup and data transfer rate is 5.63 Mb/s, but for the full 
cycle tne rate drops to 1.87 ht)/s, whicn is clearly unacceptable. To overcome 
this, many Quadrants are operated concurrently. Data are loaded into tne first 
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Figure A.14. write Seouence 

two Quaarants and tnen a 180 microseconds write comand is issued. While tnat 
Quadrant pair's MA's are in tne write mode, tne system controller loads data 
into tne second Quadrant pair, followed by tne write connand; tnen, to tne 
tniro Quadrant pair, etc. After a Quadrant pair nas completed tne write 
seouence, it is powered down and anotner Quadrant pair is accessed to replace 
it. TTiis procedure continues until all 18 Quadrants of the data file nave been 
accessed. In tnis manner, up to four Quadrants are in tne write command mode 
while tne two present Quadrants are being loaded with data. This seouence (of 
64 bits to eacn of four arrays in a Quadrant) for all eighteen Quadrants that 
store the 72-bit words, is executed 64 times. Tne total write seouence reouires 
52 ms for all 256k data bits for a data rate of 5.1 KO/s. 

4.3.3 Dual-Channel Operation 

The instruction seouences which control the memory wafer execution of . each 
operation are generated by tne nigh-speed seouencer. "H^e set of instructions 
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wnicn the nl^speed seouencer generates were listed In table 3-4. The hSS 
directs tne memory wafer commands to eacn of tne two cnannels tnrougn tne use 
of its cnannel control (CHC) flip-flops. First, one cnannei is opened by tne 
cnannei controller aaoress lock (CCL) conrand. The setting of tne CHC flip- 
flops during tnis CCL dictate wnich cnannei tne channel controller is using. 
The second channel is opened by addressing the other channel controller, via 
the CC'^, with tne CHC flip-flops placed in the opposite setting. 

Then the CHC flip flops are alternately set to indicate cnannei 1 or channel 2 
cormunication, as shown in figure 4-15. The Quadrants receive instructions 
alternately since one is in a required power«<jp delay mode; this decreases the 
resultant setup time. Once all eigpt MA's of tne two Quadrants nave been ad- 
dressed and are fully powered, both cnannels are accessed sintjltaneousiy to 
execute the data-related instructions, by supplying tne two CHC signals to- 
gether. The procedure of addressing a second Quadrant while tne first is 
waiting for a component to power up, and then simultaneously moving data in/out 

of both Quadrants, achieves the required system data rate with more efficiency. 

— ,1^^^^^^^^^— 
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Figure snows a read seouence Deing executed In only 64 microseconds, mucn 
less tnan tne 91 microseconds maxiffum reouired for tr>e data rate. Tnis excess 
time allows. tne HSS and system microprocessor to exert better control over tne 
system. 

The wafer instruction sequence for the write and erase operations remains 
basically tne same as the read seouence. The H6S issues the same addressing 
seouence, multiplexed between the two channels, for all three operations. Only 
tne data seouence, directed to both channels, differs in the three operations 
as indicated by figure 4*16. The main difference between the write and erase 
sequences, and tne read sequence, is that due to tne extended write and erase 
operations, the addressing and data subsequences overlap the actual erases and 
writes. 

4.4 K€MCFY SYSTEM CAPACITY EXPANSION 

The memory system capacity can be altered in two increments. T^e larger incre* 
ment is 107.4 ^Oits of data which constitute a full canister of memory wafers 
(256 wafers). The smaller increment is the 13.4 ►tolts which make up a power 
group of 32 wafers. There are eight power groups in a canister. 

A full canister contains 107.4 ht)its for data, 14.2 ^t)its of error code, and 
6.4 Nt)its of spare memory. A total of 8 canisters can be incorporated into the 
system, for a total of 1.024 Gbits of memory, before the on-wafer address 
space of the system is exceeded. Although more than el^t canisters can be used 
in a system, it requires that the power switching become a part of the wafer 
addressing. Qianging the norber of canisters nas little affect on tne speed or 
power specifications. 

Tne smaller memory capacity increments can be based on power groups. A power 
group contains 13.4 K3its of oata, 1.8 KOits of error code, and 0.8 Nbits of 
spare memory. A memory system based on power group units can either use a 
different size of canister or partially filled canisters for packaging. 
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Cnanging tne memory size affects tne nurcer of Bus drivers, power swi tones, 
etc. It does not greatly inpact tne system controller. The Quadrant power 
group table is tne nriost obviously affected section of the system controller, 
since it will nave more or fewer entries depending on tne size of tne memory. 
The size of tne RAM required to store tne table in tne system microprocessor 
w;..^ also cnange accordingly. 

Memory capacities exceeding 1.024 Gbits will reouire riestructuring of tne file 
address tables. Major increases in tne memory capacity also affect tne memory 
system interface since more bits must be added to tne data file nuTt)er supplied 
to tne system. Reconfiguration is also affected by memory capacity changes, 
since more memory must be searched to find a spare during major reconfiguration 
to replace entire Quadrants. 

The smallest amount of memory wnicn would make a reasonable system is five 
wafers. This is a 2.1 Kbit system. Five wafers are necessary since 18 Quad- 
rants are required for a single 256 kbit data file. However, with memory sys- 
tems of ouch less than 107 Nbits, one canister, the system controller begins 
to snow an overkill of capability. 

4.5 MB4DRY SYSTEM SPEED EF«ANCO€NT 

Tne speed of tne memory can be increased by operating additional Quadrants in 
parallel. The two major effects of increasing tne speed are the system power 
reouirements and the modification of tne system controller software. Minor 
nardware modification is necessary at tne interface to the memory wafers to 
multiplex and control tne flow of data from multiple ports. Eacn of these 
ports would correspond to a separate active Z-txis. Each is connected to a 
different canister to operate separate pairs of channels. The memory system 
could then operate at an additional 5 Nto/s for eacn added port. 

Tne aooition of eacn memory wafer port would result in approximately 12. 5W of 
additional peak power (9.2W average power) for a read operation and approxi- 
mately 23. 4W additional peak power (20.7Vr average power) for write operation. 
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Tnus, a mocifieo memory system operating at 15Mb/s would reouire approximately 
peak power (77. average power) during a write operation. 

For memory system operation mucn above 15 W)/s, the speed capaoility of tne 
system controller and error detection/correction circuitry (EDAC) would need 
to De irrreasec. First, tne speed capadility of tne CM0S/SC3S EDAC would be 
surpassed, and Dipolar components sucn as ECL, would be used. TMs would result 
in a power increase of about 5 to 7 watts. Snortly tncreafter, tne spee6 capa- 
bility of tne nign-speed seouencer would be exceeded and would nave to be modi- 
fied into a "multi-sequencer." Tne multi-seoucncer version of tne nign-speed 
sequencer would require an additional Advanced Micro Devices 2910 rrdcro- 
sequencer and faster memory for a power increase of approximately A to 6 watts. 
In any case, tne maximum speed of operation of sucn a modified memory system 
is estimated to be 100 Mb/s using SOOW of peak power during write operations. 
Otner organizational approaches snould be investigated for significantly ni^r 
speed systems. However, system design cannot overcome tne basic speed-power 
properties of electronic systems. 
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Section 5 
SYSTEM CONTROLLER SOFTKHARE ROUTINES 



5.0 SYSTEM SOFTWARE DESCRIPTION 



Tne relationsnips Between tne main routines of tne system controller corrponents 
are srown in tne block flow aiagram of figure 5-1. T^e routines snown on tne 
saro norizontal row operate concurrently. The tnree colums indicate tne loca- 
tion of tne routine: power kernel, system rrdcroprocessor, or nign-speed 
sequencer. 

Tne power kernel routine is executed, almost continuously, by eacn of tne power 
kernel processors sinultaneously . This routine checks each of tne power kernel 
processors ana the system microprocessor CRJ. When the system microprocessor 
is to be tested, tne power kernel processors issue a restart interrupt wnicn 
initiates its Initialization routine. " 
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figure 5-1. Block Flow Diagram Sxjwing Relationsnips 
Between Routines of System Software 
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Tne initialization rcutine tests and initializes tne system microprocessor, tne 
nign-speed seouencer (l*S), and tne error detection/correction circuitry 
(EDAC). After ail tests are passed, tne initialization routine accepts tne 
first system corrTTand and passes control to tne monitor routine. 

Tne system microprocessor monitor routine starts tne nigrvspeed seouencer gen- 
erating its basic seouence, as defined by tne system conrand. Tne monitor 
routine tnen cnecks tne progress of tne HSS tnrougn its basic seouence, and 
watcnes for error information from tne EDAC. Once tne nas completed its 
seouence, it generates an interrupt to tne system microprocessor, wnicn causes 
control to pass from tne monitor routine to tne completion routine. 

Tne system microprocessor completion routine first cnecks to see if tne system 
corriTBnd nas been completed, or if anotner cycle of tne HBS seouence is re- 
Quired. If tne system conrand nas not been completed, control is returned to 
tne monitor routine. Otnerwise, an interrupt is sent to tne power kernel 
causing tne power kernel processors to monitor and control a system micropro- 
cessor CRJ test. 9xuld tne CPU fail its test, system initialization is 
restarted, wnen tne CRJ nas passed tne test, tne system microprocessor accepts 
tne next system conrand and passes control to tne monitor routine. 

During read operations, tne EDAC issues interrupts to tne system microprocessor 
monitor routine wncnever it detects a data error. Upon receiving tnis inter- 
rupt, tne monitor routine passes control to tne system microprocessor data 
error routine. Tne data error routine reads tne EDAC error status and stores 
it in an error file. Tne data error routine tnen returns to tne monitor. 

After data errors nave been detected, tne completion routine will request to 
pass control to tne system microprocessor error preprocessing routine by set- 
ting tne system status reconfiguration flag. Tne system reconfiguration conv- 
mand causes tne error preprocessing routine to examine tne error status infor- 
mation in tne error file. Depending on tne information in tne error file, tne 
error preprocessing routine selects tne proper built-in test to isolate tne 
faults wnicn nave appeared in tne memory arrays and controllers. (Tee all tne 
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faults nave oeen removea, tne error preprocessing routine returns control to 
tne completion routine for tne CPU test, 

5A P0V£R KERNEL: ROUTIh€ OF PROCESSORS 

The function of a power kernel processor, processorj^, is to perform and monitor 
tests of itself, tne otner power kernel processors, and tne system micropro- 
cesssor CRJ. The execution of tnese tests form tne main routine of tne power 
kernel processor as snown in tne flow cnart, figure 5-2. 

Immediately upon receiving system power, processor^ begins execution of a self- 
test routine. TMs routine botn initializes processor^ and fully exercises it. 
The result of tnis self test is a niittoer wnicn processor^ outputs to tne otner 
procesors of tne power kernel. Processor^ compares tne results received. If at 
least one of tne results from tne otner processors matches processor^'s result, 
tnen processor^ votes to leave processor powered. If tne majority of otner 
processors also vote to leave on processor^, the next processor, processor^^^ , 
will nave its results voted upon. Sould a majority of processors disagree witn 
processor j^, tnen processor^ is powered down. 

vTien processorj^ is powered down, as a result of tnis voting procedure, its re- 
placerent is powered up. Tnen tne self-test and voting procedure is restarted 
Dy all tne power kernel processors. 

After, eacn of tne power kernel processors nas nad its self-test results suc- 
cessfully Checked* and passed, a test of tne system microprocessor CPU is be^n. 
Tne status of the CRJ is read and tne currently active system microprocessor is 
found. Processori tnen issues a power-on signal to tne CPU wnicn is voted on 
witn tne rest of tne power kernel processor outputs. Once power is applied, 
processori issues a reset interrupt to tne system microprocessor CRj, wnicn is 
voted upon. This causes initialization of tne system microprocessor and tne 
start of a CPU self-test. Processor^ tnen waits for tne results of tne system 
microprocessor self-test. Should these results not be received within a given 



5-3 



LEX15531 



voiLme ill 




Figure 5-2. Pomet Kemei Routine of Processor^ 



5-4 



LEX15532 



^iLine III 

time limit, processorj^ votes to turn off tnis current CRJ. when tne test re- 
sults are received in tiro, processorj^ conpares tncse witn tne known correct 
results of tne test. If a majority of tne power processors agree on tne posi- 
tive results of tne test, tnen tne bootstrap of tne current system micropro- 
cessor is started* Snouid tne majority of the power processsors agree tne cur- 
rent system microprocessor CPU fails tne test, it is powered down and tne spare 
CRJ next in line is powered up and tne test procedure resLrad. 

Once tne power processors agree tnat tne system microprocessor nas passed tne 
CRJ test, tne CRJ must be re tested periodically to prevent a malfLTCtion from 
occurring witnout being detected. The system microprocessor initiates tnis 
test between file operations; nowever, since a file operation may never end if 
tne system microprocessor fails, tne power kernel perfozms a time-out to detect 
tnis failure. 

After initiating tne system bootstrap, processor^ will zero a counter. Between 
each power kernel self-test this counter is incremented and tested. Sxiuld the 
counter of processor]^ reach the allowed maxinun, processorj^ will begin tne CRJ 
test routine by setting tne system microprocessor turr>-on flag. Initialization 
and test of tne system microprocessor CRJ then proceeds as during the initial 
power up. This system microprocessor tum-on flag can also be set from outside 
tne memory system, snouid the outside world detect a system microprocessor 
failure first. 

5.2 RDV€R KERrCL: INTERaPT ROUTIhC 

After completing a file operation, tne system microprocessor issues an inter- 
rupt to tne processors of tne power kernel. This interrupt notifies tne power 
kernel tnat an operation has been completed, and tnat the system microprocessor 
now nas time for a snort self test, upon receiving this interrupt, the power 
kernel processors stop executing tne main routine given in figure 5-2, and 
begin execution of tne power kernel interrupt routine, flow charted in figure 
5-3. 
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Tne interiLpt routine Is similar to tne system microprocessor CRj test of tne 
basic processor routine. SxxjIc) processorj^ determine tne system microprocessor 
nas passed tne CPU test, processori will vote to pass tne CRJ. If a majority 
agree tnat tne CRJ passes, tnen normal operation of botn tne power kernel and 
tne system microprocessor continues. hDwever, If a majority agree tnat tne CRj 
nas failed its test, tnen tne system microprocessor turrvon flag is set and tne 
full system microprocessor self test is executed by tne power kernel as in tne 
main processor routine. 

5.3 SYSTEM MICRCPRQCESSQR: INITIALIZATION ROUTIIC 

Tbe system microprocessor is powered up by tne power kernel. Initially upon 
receiving power, only tne CRJ and a small section of ROM are active. Tne CRJ 
tnen executes a self test routine wnicn resides in tne active RCM. After com- 
pleting tnis test, tne power kernel cnecks tne results. . It tnen eitner enables 
tne remainder of tne system microprocessor initialization routine, or powers up 
anotner CRJ module. Power is received by tne rest of tne system after tne CRJ 
passes its self test. 

Once tne power kernel decides tne CPU is properly functioning, tne CRJ will 
continue testing tne rest of tne system microprocessor, to bootstrap itself to 
a fully functional level, as snown in figure 5-4. First tne microprocessor CPU 
will cneck out tne rest of its memory. Sxxild tne CPU decide tnat tne memory 
is not properly functioning, it will switcn in a spare memory module. After 
finding a functional memory module, tne CPU tests its I/O controller and I/O 
sections. If tne CRJ decides tnat tne I/O section is not fully operational, 
it tnen switcnes in a spare I/O mocxjle. Oicc a properly functioning unit is 
found, tne system microprocessor configuration is complete. 

Tne system microprocessor next tests tne nign^speed sequencer, "n^e system 
microprocessor enables tne HSS to execute a self test routine. As tnis routine 
progresses, tne system microprocessor monitors tne progress of tne H6S by 
reading tne HSS-system microprocessor status word. If tne HSS acnieves its 
different states in tne proper order, tne system microprocessor decides tne hBS 
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Figure System Microprocessor Initialization Routine 
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is fault free* However, snould tne hss acnieve an improper state, tne system 
nrdcroprocessor analyzes tne test state seouence to deduce wnlcn hSS module is 
faulty. Tne system microprocessor tnen causes tne faulty HSS subsection to be 
replaced by a new module, and tne h6S self test is restarted. 

Finally, tne error detecting/correcting circuitry is tested. The EDAC is 
placed in tne encoder test mode oy tne kBS to set up tne proper data patns. 
The system microprocessor tnen uses tnese test data patns to supply tne EDAC 
witn a known seouence of data, vren tnis data, now encoded, is returned to tne 
system microprocessor, tne system microprocessor cnecks tne encoding of tne 
data witn proper encoding for tne data. If an error occurs, tne system micro- 
processor isolates tne faulty EDAC module wnicn caused tne failure, and 
replaces it witn a good spare. Tne encoder test is tnen repeated. Orce tne 
EDAC passes tne encoder test, tne HSS svitcnes tne EDAC to tne decoder test 
mode. The system microprocessor tnen sends encoded data, wnicn contains some 
known errors, tnrou^ tne EDAC, Botn tne output of tne EDAC and its error 
status word are monitored by tne system microprocessor. Sxxild an unexpected 
error be indicated by tne error status word, or an expected error not snow up 
in tne error status word, a fault is indicated. When a fault is found, tne 
system microprocessor analyzes botn tne decoded data output and tne error 
status information to identify the location of tne fault. Any fault found by 
tne decoder test will be resolved by switching in a spare EDAC module to 
replace tne malfunctioning section. After rectifying a decoder fault, tne sys- 
tem microprocessor restarts tne test of tne EDAC encoder. 

Or\ct tne EDAC passes the system microprocessor's testing, tne system micropro- 
cessor refreshes tne address tables stored on the wafer. Then tne system 
microprocessor updates the system status to snow tne memory system has finished 
initialization. The system microprocessor tnen requests to perform a test of 
all tne memory wafers, known as tne periodic built-in test. If permission is 
granted, tne system microprocessor executes tne test, and tnen enter tne system 
microprocessor completion routine. Bermission is not granted for the periodic 
built-in test wnen there is anotner corrrand for the memory system to execute. 
This cariTBna is decoded and tne h6S is initialized to execute its part of tne 
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cofTTTBOd. Tne system microprocessor tnen begins execution of tne system com- 
mana Dy entering tne system microprocessor monitor routine. 

5.4 SYSTEH MIO^CPROCESSCR: MONITOR ROUTIhC 

vvhile tne nign-speed sequencer is issuing instructions to tne wafer and tne 
error detecting/correcting circuitry is encoding/decoding tne data, tne system 
microprocessor monitors tne functioning of eacn to ensure tney are executing 
tne proper routines, Tne monitor routine flow cnart snown in figure 5-5 
illustrates tne metnod of accomplisnment. 

A system command is initiated by enabling tne HSS, wnicn nas been previously 
set up to issue tne sequence of wafer instructions for tnat system cornmnd. The 
system microprocessor interrupt logic is tnen enabled so tne microprocessor can 
receive any critical information from tne rest of tne memory system: wafer 
addressing errors,. EDAC data errors, HSS completion errors, etc. 

The system microprocessor monitors tne progress of tne h6S by reading tne H5S 
system microprocessor status word. The system microprocessor tnen decodes tnis 
status word into a state nuiber, for tne current system cormend. Tnis new 
state nLTToer is cnecked by looking in tne HSS state table, for tne current sys- 
tem command, to see if it is a legal state norber. If tne new state nurder is 
legal, it is loaded into tne current state counter. If tne new state ncfTt>er 
is illegal, tne state naitoer calculations are repeated to ensure tnat tne sys- 
tem microprocessor is not at fault. Snould tne new state nurder still be 
illegitimate, tne >SS is disabled and tne system status word is cnanged to 
indicate a system nalt due to a severe failure. An HSS dia^ostic is tnen 
executed. 

Once tne system microprocessor nas concluded tnat tne HSS nas properly pro- 
gressed to a new state, it cnecKs to see if a new system conmand nas been 
issued. If no new system conrond nas been issued, tne system microprocessor 
repeats its cneck of tne K6S state. 9xuld a new system commnd occur, tne 
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Figure 5-5. System Microprocessor Monitor Routine 
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sysiem microprocessor naits tne h6S and sets it up to issue tne ne* sequence of 
wafer instructions. TT^en system microprocessor restarts tne monitor routine 
as if it nad Just been initialized. 

Tnere are t»o system microprocessor interrupts wnlcn may occur during normal 
execution of a system conrond. First is tne completion interrupt from tne H6S 
wnicn indicates tnat an HSS seouence is about to be completed. Tnis interrupt 
causes an exit to tne system microprocessor completion rtjutlne. "me otner 
interrupt is tne data error interrupt from tne EDAC wnicn Indicates an error 
nas oeen detected in tne data being decoded to make error status information 
available. Tnis interrupt causes an exit to tne system microprocessor data 
error routine. 

5.5 SYSTEM MIWCPRXESSOR: COKPLETION ROUTIhE 

v«hen tne h6S is about to finisn an instruction seouence, it notifies tne system 
microprocessor by issuing a corrpletion interrupt. Upon receiving tne interrupt, 
tne system microprocessor enters tne completion routine, as flow cnarted in 
figure 5-6. Tne first function of tne routine Is to decide if tne system com- 
oand nas been car^leted or if tne H5S taist repeat tne instruction sequence at 
least once more. To repeat tne HSS sequence, tne system microprocessor 
re-enables tne ^6S and returns to tne monitor routine. 

Snouid execution of tne system commnd be completed, tne systan microprocessor 
updates tne system status to Indicate conrend flnlsned. "men tne error counter 
is cnecked. If tne error counter is not zero, tne system microprocessor up- 
dates tne system status word to reconfiguration. The external system tnen 
allows reconfiguration of tne memory system, if feasible. If reconfiguration 
is allowed, tne system microprocessor tnen enters tne error processing routine. 
After coTvieting error processi^ng, or if no reconfiguration was done, tne sys- 
tem microprocessor Interrupts tne power Kernel, to indicate tnat tnere is time 
for a systan microprocessor CM test. Tne power kernel controls tne executing 
of tne CPU test. 3x)uld tne CPU fail, tne power kernel maintains control of 
tne systan microprocessor and performs a complete system microprocessor test, 
as done during system initialization. 
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Figure 5-6. System Microprocessor Oarrpietion Routine 
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men tne system microprocessor passes tne CRj test, it first cnecks for a new 
system canmand. If a new system corrrand nas Deen issued, tne command is decoded 
ana tne HSS set up to execute tne conrond, Tne system microprocessor then 
enters tne monitor routine. The system microprocessor performs a bootstrap to 
test tne entire system controller, as in the system initialization, if no new 
system conrond nas been issued. 

5.6 SYSTB^ MIORCPRQCESSOR: DATA ERROR RCXJTIhC 

When a data error occurs during a read operation, tne error detecting/correc- 
ting circuitry notifies the system microprocessor tnat an error nas occurred. 
Thus, tne error status information can be retrieved by tne system microproces- 
sor before it is lost. The EDAC notifies tne system microprocesor by issuing 
a data error interrupt. This interrupt causes tne system microprocessor to 
enter tne data error routine whicn retrieves and stores tne error infozmation. 
Some simple processing is done on the error data to save storage space. The 
routine is flow charted in figure 5-7. 

Upon receiving tne interrupt, the system microprocessor disables tne inter- 
rupts, retrieves tne error status information, and then re-enables tne inter- 
rupts. By not enabling tne interrupt logic until after tne error status has 
been retrieved, tne system microprocessor guarantees tnat the error status is 
not lost due to delays caused by otner interrupts. After tne error informa- 
.tion is retrieved, tne error counter is incremented and.tnen the error status 
is stored away. 

When the first error occurs, an error data file is set up to store tne error 
aata in a more useful simplified form. If the error is a nviltiple-bit error, 
the error status is stored in tne multiple bit error file section and the 
fTultiple error counter is incremented. Since fojltiple bit errors are not 
correctable by the EDAC, the system status word is updated to inoicate tnat 
bad data exists in tne file. 
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Figure 5-7. System Microprocessor Data Error Routine 
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Since singie-Oit errors are correctea oy tnc EDAC, tne system status need not 
oe cnanged wnen tney occur. The location of tne faulty dit, bit nortjer, is 
located and its file area found. In tne file area of tne bad bit nurtoer, tne 
system buffer address word nortoer is also stored. If tne word nortoer nas been 
stored in tnis file area before, tne associated counter is incremented to snow 
tne niiToer of repeats of tnis bad bit-^ord noitoer corrdination. If tne word 
nuiTDer nas not been stored before, it is loaded witn a zero count to snow a new 
Dad bit-word norter corrdination. TMs bit-word nuitoer corrdination is used to 
decide wnetner a permanent or a transient failure caused tne errors. 

SxDuld tne error counter indicate more tnan tne allowable nunber of bits in 
error, a failure in eitner tne EDAC or tne HSS is assured and a severe test is 
perfoirred. TMs severe test nalts tne system and indicates a nalt Aje to 
failure in tne system status, A test of tne EDAC is tnen initiated. Any 
faults located by tnis test are rectified by replacing tne bad EDAC modules 
witn standby spares, and tne test is restarted. Should tnis EDAC module 
replacement not rectify tne situation, tne system microprocessor assures tne 
HSS is at fault and conducts a test of it. After tne faults nave been removed, 
tne file operation is restarted. 

5.7 SYSTEM MICPCPROCESSOR: ERROR PREPROCESSING ROUTIhE 

vr^enever errors occur during a read operation, tne error preprocessing routine, 
snown in figure 5-^, is entered by tne system microprocessor at tne conclusion 
of tne read operation. The error preprocessing routine decides tne best built- 
in test for tne type of errors recorded. 

Tne first step toward executing a built-in test (BIT) is to reread all tne data 
of tne file to remove all transient errors. Any error not exactly duplicated 
during tne reread is considered transient. Should tne reread snow tnat no 
static errors nave occurred, a BIT is no longer necessary and tne error prepro- 
cessing routine is exited. If only multiple bit errors appear permanent, tnen 
tne file level BIT is executed. TMs level of BIT tests tne entire data file 
to locate tne faults, since multiple bit errors cannot be isolated by tne EDAC. 
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Figure 5-8. System Microprocessor Error Preprocessing Routine 
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Once tne fault areas are located, tne system microprocessor reconfigures tne 
affectea areas and retests to see if tne faults nave been corrected. (J«e tne 
faults are removed, tne routine is exited. Snould BIT not rerove tne faults, 
testing continues until error-free operations are possible. 

When single-bit errors are detected, more information is available on the loca- 
tion and nature of tne fault, so xess drastic BIT'S are tnen possible, "me 
first level of testing is to check for groups of 4 bits which have been in 
error. These bits, if properly aligned, can signify a channel controller fail- 
ure, and so the CC-level BIT is executed. Upon completion of the CC-level BIT, 
the data is reread to discover which faults nave been resolved by the CC test. 
Next, the remaining error bit locations are screened for pairs which are prop- 
erly aligned to indicate an AC failure. v«hen such bit pairs are found, an 
AC-level BIT is executed. The faults removed by the AC test are tnen found and 
eliminated. 

The remaining errors should only be random single-bit errors and their associ- 
ated coincident tiultlple-bit errors. For each slngle-tolt error remaining, a 
memory array-level BIT is executed on the specific memory arrays. Should any 
faults appear to remain after this final sub-level of testing, a file-level BIT 
must be executed to resolve the problem. 

5.7.1 CC-Level BIT 

When tne system microprocessor error preprocessing routine determines tne type 
of error detected reserroles a channel controller failure, it calls the channel 
controller (CC) level BIT. The channel controller level BIT is corposed of 
three test seojences: channel controller component test, channel controller 
comparison test, and replacement test. Together these three test sequences are 
used to verify tnat a suspect CC is really faulty, and that replacing it with 
a spare actually removes tne fault. The relationship between tne three test 
seouences is snown in the flow cnart of figure 5-9. 
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Figure 5-9. CC-Levei BIT 
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T^e first test seouence of tne CC level BIT is tne CC corrponent test. iMs 
test secxience attempts to verify a CC failure by testing individual components 
inside tne CC, A set of test steps is executed by tne CC; eacn test step ver- 
ifies one or more ftrctions of a key component. As tne CC passes eacn test 
step, tne function tested is assoned to be operable and can tnen be used to 
test anotner part of the CC» e.g. » after tne spare row counter (SRC) passes 
its test steps, it can be used to read tne test data from tne array controller 
map (ACM). The individual test steps of tne CC component test are listed in 
table 5-1. 

Snoulo cne CC fail any of tne test steps twice in a row, tne CC is assured to 
be faulty, indicating tnat tne problem nas been found. To verify tnat tne CC 
alone nas been causing tne detected data errors, tne alternate CC is switcned 
into tne Quadrant's configuration and its operation is verified by executing 
tne test steps of Table 5-3. Tne replacement test is tnen executed. The 
replacement test is tne final cneck. 3r3ul6 tne quadrant pass tne replacement 
test, tne quadrant is assuned to be fault free. The test steps of tne CC 
replacement test are detailed in table 5-2. 

After tne questionable CC passes eacn test step of tne component test sequence, 
tne alternate CC is addressed and tne cnannel controller comparison test 
sequence is executed. However, before tne cnannel controller comparison test 
can be executed, tne alternate CC must be cnecked to be sure it is functioning 
properly. This is done by executing tne CC verification test. Tne CC verifi- 
cation test is a < subsequence of tne CC component test sequence wnicn performs 
every test step of tne component test except any step wnicn alters nonvolatile 
memory. E&cn test step of tne verification test is given in table 5-3. 

Once tne alternate CC nas nao its operaoility verified, tne CC comparison test 
is performed. The comparison test compares tne contents of eacn of tne NVM 
address maps. By performing a location-oy-location comparison of tne maps, and 
tnen analyzing every discrepancy, tne differences can usually be attributed to 
a fault in one of tne CC*s. In tnis case, tne CC wnose map is believed to be 
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TABLE 5-1. CC QDNPOI^T TEST 

1. Address Questionable CC and wait for CC acknowledge. 

2. Test SRC Dy incrementing it from one zero test to anotner, cnecking tne 
noToer of incretrents needed. 

3. Test SRC, Y-Dus, and M/L muxes Dy loading known bits from Z-bus into SRC 
and incrementing SRC to zero as a cneck on tne load. 

4. Read tne known code words from tne special AO locations into SRC and in- 
crorent SRC to zero to cneck tne AO read, and tne ACM data retention. 

5. Read eacfi of tne addresses of tne faulty data block, botn tne row and array 
pair addresses, from tne ACM to tre SRC. Increment the SRC to zero for eacn 
address, ana store values in system microprocessor, cnecking eacn for self- 
consistency. 

6. Erase tnese two words of tne AO and tnen reread tnese words into tne SRC, 
incrementing tne SRC to zero to verify tne erase. 

7. Rewrite tne addresses back into tne AO by incrementing tne SRC to tncval- 
ues stored in tne system microprocessor and loading tnem into tne memory 
latcnes. Tnen reread eacn address back into tne SRC, incrementing tne SRC 
to zero to verify tne write. 

8. Read tne value of eacn AO location into tne SRC, incrementing tne SRC to 
zero for eacn, and loading tne value into tne system microprocessor. Then 
cneck tne self-consistency of eacn value— duplicate addresses, out-of- 
bounds aooresses, rows listed botn as spare and as allocated, etc. 

9. Test tne temporary aooress register by incrementing it from one zero test 
to anotner, cnecking tne nurder of increments used. 

10. Read tne known code words from tne active AC address map (AAA) to tne tenv* 
porary address register (TAN) and increment tne TAN to zero to cneck tne 
TAN load, AAA read, and AAA data retention. 

11. using tne addresses supplied by tne AO, read tne two locations of tne AAA 
used oy tne bad data blocks into tne TAN, incrementing tne TAN to zero and 
storing ootn values in tne system microprocessot. Tnen erase tnese two 
locations of tne AAA, After botn erasures, read tne value of eacn location 
into tne TAN, incrementing tne TAN to zero to verify eacn erasure. 

12. Rewrite tnese values back into tne same locations of tne AAA by increment- 
ing tne TAN to tne values stored in tne system microprocessor and tnen 
using tne addresses supplied by tne AO, perform eacn write. After re- 
writing botn values, reread tne two locations back into tne TAN, increment- 
ing tne TAN to zero to verify eacn write. 

13. Read tne contents of eacn AAA location Into tne TAN, Incrementing tne TAN 
to zero after eacn read, storing tne values In tne system microprocessor. 
These values are tnen cnecked If In bounds. 

14. Set and tnen clear tne power bit, latcnlng tne zero test to verify tne 
power bit operation. 
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TABLE 5-2. CC REPLACEhCNT TEST . 
1. Address alternate CC and wait for CC acknowledge. 

2 Verify operation of alternate CC. 

3 Read tne value of eacn ACM location into tne SRC, increnienting tne SRC to 
zero for eacn, and storing eacn value in tne system microprocessor. T^en 
cneck tne self-consistency of tnese values. 

4. Coi^are eacn location of tne two AOH's stored in tne system microprocessor, 
noting ano storing tne differences oetween tne two compared values. Analyze 
tne caT¥)lete set of differences to determine tne type of fault causing tne 
difference and wnicn CC is at fault. 

5. Read tne value of eacn active AC address location into tne TAN, using tne 
addresses supplied by tne A04. Increment tne TAN to zero for eacn value, 
storing eacn value in tne system microprocessor. Check eacn value for 
validity. 

6. Compare eacn value of tne two active AC addresses stored in tne system 
microprocessor, noting and storing any differences. Then analyze tne set 
of differences to determine botn tne type of fault and tne CC causing tne 
difference. 



TAB-E 5-3. CC VERIFICATION TEST 

1. Address CC and wait for CC acknowledge. 

2. Test SflC by incrementing it from one zero test to anotner, checking tne 
nijitoer^of increments used. 

3. Test SRC and Y-bus and M/L muxes by loading known bits from X-bus into SRC, 
both MSH ano then LSH, incrementing SRC to zero as a check on the load. 

u Read tne known code words from tneir special AW locations into SRC, and 
incrorent tne SRC to zero to check tne ACM read and the ACM data retention. 

5. Test tne TAN by incrementing it from one zero test to another, checking the 
noToer of increments, used. 

6. Read tne known code words from tne active AC address to tne TAN and incre- 
f7«nt tne TAN to zero to check the tan load, active AC address read, and 
active AC address data retention. 

7. Set and clear tne power bit, watching the zero test to verify tne power bit 
operation. 
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caiiea. The array controller level BIT is as similar to tne CC-ievei BIT as 
tne AC Hardware is to tne CC Hardware. The AC level BIT consists of tnree test 
seouences, as did tne CC level BIT; AC component test, AC comparison test, and 
replacement test. Togetner tnese tnree tests verify tnat a Questionable AC is 
faulty, and tnat replacing it actually removes tne fault, me way tne tnree 
test seouences cooperate is flow cnarted in figure 5«10. 

The first difference between tne AC level BIT and tne CC level BIT is since a 
CC must be addressed before an AC can, tne CC of the Quadrant is addressed and 
verified. Oxe tne operability of the CC nas been verified, tne AC under Ques- 
tion executes tne AC component test steps. The AC component test seouence is 
basically tne same as tne CC component test;, tne individual test steps are 
given in table 5-A. Should the AC fail to pass any of tne test steps, an 
alternate AC is switched into tne Quadrant configuration and the replacement 
test is executed to verify that replacing tne AC removes tne fault. 

Qrtce the AC completes tne AC component test without failing any test step, an 
alternate AC is addresed and tne comparison test (table 5-5) is executed. 
Sxuld tne analysis of tne AC comparison test fail to indicate clearly an AC 
failure, tne third AC is addressed and tne comparison test is repeated, with 
tnree complete sets of values, tne AC comparison test can tnen indicate which 
AC is faulty. Then the replacement test can be executed with a good AC and 
removal of tne fault can be verified (table 5-6). 

Should tne replacement test indicate tnat replacing the Questionable AC does 
not remove the fault, the error preprocessing routing will tnen determine 
whether the CC level BIT or the memory array level BIT is executed next. When 
less than tnree AC*s are available for the comparison test, the comparison test 
using only two AC*s is used. Should the comparison test not be possible because 
there are no spares available, tne burden of removing the fault is placed upon 
the replacement test under control of tne system microprocessor error prepro- 
cessing routine. 
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TASLE 5-A. AC aDM=ONENT TEST 
. Address QuestionaDle AC and wait for AC acknowledge. 

2 Test spare array counter by incrementing it from one zero test to anotner, 
cnecking tne nuroer of increnents used. 

3. Read tne known code worts from tneir special locations in tne array map 
into tne spare array counter, and increment tne spare array counter to zero 
for eacn to cf«ck tne spare array counter load, tne array map read, and tne 
array map data retention. 

4. Read eacn of tne addressed used by tne faulty data block from tne array map 
to tne spare array counter. Increment tne spare array counter to zero for 
eacn address, storing tne values in tne system microprocessor, and cnecking 
eacn value for self consistency. 

5 Erase tnese two worts of tne array map, tnen read eacn into tne spare array 
counter, incrownting tne spare array counter to zero to verify tne erase. 

6 Rewrite tne values back into the array map by incrementing tne spare array 
counter to tne values stored in tne system microprocessor and perform eacn 
write. Then reread eacn value back into tne spare array counter, incre- 
menting eacn to zero to verify tne write. 

7 Read tne value of eacn array map location into tne spare array counter, 
incrarenting tne spare array counter to zero for eacn and storing eacn 
value in tne system microprocessor. Tnen cneck tne self-consistency of 
eacn value—duplicate addresses, addresses out-of-bands, addresses listed 
as botn allocated and spare, etc. 



TABLE 5-5. AC COMPARISON TEST 

1. Address next alternate AC and wait for AC acknowledge. 

2. Verify operation of alternate AC. 

3. Read tne value of eacn array map location into tne spare AC counter, in- 
crerenting tne spare AC counter to zero for eacn and storing eacn value in 
tne systwn microprocessor. Then cneck tne self-consistency of tnese values. 

4. COnpare eacn value of tne two array maps stored in tne system micropro- 
cessor, notiro and storing tne differences between eacn pair of compared 
values. Analyze tne catplete set of differences, botn tne type of fault 
and wnicn is at fault. 

5. Snould analysis of tne array map discrepancies not yield a clear failure 
TOde, tne tnird AC is addressed and tne comparison of array map values is 
repeated. Tnis tnirt set of values is analyzed to identify wnicn AC is 
faulty. 
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TABLE 5-6. AC VERIFICATION TEST 

1. Address AC and wait for AC acknowledge. 

2. Test spare AC counter by incrementing it from one zero test to anotner, 
cnecking tne norder of increments used. 

3. Read tne known code words from tneir special locations in tne array map 
into tne spare AC counter » and increment tne spare AC counter to zero for 
eacn to cneck tne spare AC counter load, trie array map read, and tne array 
map data retention* 

5.7.3 MA-Level BIT 

Should tne system microprocessor error preprocessing routine determine the type 
of error detected is most likely caused by a memory array failure » the memory 
array level BIT is executed. The memory array level BIT is composed of the 
memory array component test sequence and the replacement test sequence, as 
Shown in figure 5-11. 

Before an MA can be accessed, the CC and AC of its quadrant must be accessed, 
SO both a CC and an AC are addressed and operationally verified before the MA 
component test sequence is executed. The MA compxinent test sequence not only 
cnecks the operation of the MA itself, but also checks the operation of eacn of 
the eignt data lines. The check of the data lines is accomplished by shifting 
data in and out of tne MA snift registers eight times, once over each data 
line. Should tne data returning over one of the data lines differ from the 
data of tne others, tnat data line is assaned to be bad. This data line failure 
is then verified by switching to another row and repeating tne test. From this 
repeated test, the location of the data line fault can be deteimined and the 
action to be taken (abandonment of a row or quadrant) confirmed. The individual 
test steps of this sequence are listed in tables 5-7 and 5-8. 

Whenever an MA which contains data is addressed as the result of a test, every 
attempt is nnade to save tne data stored there, l^is begins with the original 
reread of the faulty data performed by the error preprocessing routine. Each 
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TABLE 5-7. MA aD^«)^eNT TEST 

1. Address Questicjnable MA and wait for MA acknowledge. 

2. Load word aooress register witn ail ones and tnen ail zeros and tnen ail 
ones again, watcning tne zero test signal to verify zero loading. 

3. tepeatedly load tne word address register witn only one bit set to one, 
sniftif^ tne location of tne set bit between every load and watcning tne 
zero test line to verify eacn bit is set. 

4. snift sixty-four known data bits Into tne MA snift registers, and tnen 
snift tne data back out of tne snift registers, cnecking to make sure tne 
sane data returns. 

5. Readoress tne CC, AC, and MA of tne Quadrant, cnanging tne data lines used 
eacn time. After eacn readdressing, snift tne same known data in and out 
of tne MA snift registers watcning to be sure tne same results are received 
after eacn snift test step. 

6. Snould any data stored in tne MA not be stored in tne system microproces- 
sor, attaipt to read tne data out of tne MA and store it in tne system 
microprocessor Rtt^, cnecking tne daU to be sure tnat it is reasonable. 

7. Erase tne MA ano tnen read tne contents of tne MA to verify tne erasure. 

8. write a known test pattern into tne MA and tnen read tne test data out of 
tne MA to verify tne write. 



TPELZ 5-8. VERIFICATION TEST 

1. Address MA and wait for MA acknowledge. 

2. Load tne word address register witn all ones and tnen all zeros and tnen 
all ones again, watcning tne zero test sigial to verify zero loading. 

3. Repeatedly load tne word address register witn only one bit set to one, 
' snifting tne location of tne set bit between every load and watcning tne 

zero test line to verify tnat every bit gets set. 

4. 3iift sixty-four known data bits into tne MA snift registers and tnen snift 
tne data oack out of tne snift registers, cnecking to make sure tne same 
data returns. 

5. Readdress tne CC, AC, and MA of tne Quadrant, cnanging tne data lines used 
eacn time. After eacn readdressing, snift tne same known data in and out 
of tne MA snift registers watcning to be sure tne same results are received 
after eacn snift test step. 
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set of bits wnicn are detected to be erroneous are stored Dy tne system micro- 
processor as tney are corrected. During eacn subseouent reread of tne flle» 
tnese Dlts are stored again snould tne removal of a fault provide additional 
correctaoility. Once a faulty MA is replaced by a good spare, tne data is 
stored into tnis HA and tne file data is once again error-free, me only «ay 
tnat data is lost is if tne errors in a file are too nunerous or tne ntirter of 
iterations tnrougn tne test sequence is large enougn to exceed tne amount of 
system microprocessor RAM available. 

Should tne MA component test seouence and tne replacement test sequence be un- 
able to succesfully verify and remove tne faulty MA, tne error preprocessing 
routine tnen determines wnetner tne CC level BIT or the AC level BIT is exe- 
cuted next. 

5.7.4 File-Level BIT 

Should a full iteration of tne CC level, AC level, and MA level BIT cycle not 
succeed in removing fault from a data block, or should there not be enougn 
information to isolate a single data block, the file level BIT is executed. 
The file level BIT is tne ultimate effort in fault removal, and as such is not 
executed except wnen absolutely necessary. The file level BIT is tne only test 
wnicn generally aestroys data stored in MA*s; this is because it tests so many 
MA'S tnat the data cannot be stored in tne RAM available. An alternate tecN. 
nique of storing data in spare MA's may be used to save this data. 

The file level built-in-test uses all tne test sequences of the btner built-in- 
tests, but generally relies on tne replacement test sequence flow charted in 
figure 5-12. The replacement test seouence accesses each component, CC, AC*s, 
and MA'S, of a data block and verifies eacn one after addressing it. The veri- 
fication test scQuences arc detailed in tables 5-3, 5-6, and 5-8. Should a 
component of tne data block not pass tne verification test, tne replacement 
test Switches to tne nearest spare and repeats tne test sequence. If there is 
no spare available or tne available spare cannot be verified, tne appropriate 
component test is executed to oetermine wnetner a row or Quadrant must .be 
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switcned. After all tne corrponents of tne oata dlock nave deen aaaressed and 
verified, tne data is reread from tne data block, wnetner it is actual data or 
test data, and tne data is cnecked for faults. If no data errors are detected, 
tne BIT of tnat oata block is corrplete. Sncxjid errors be detected, tne built- 
in testing continues witn anotner level of BIT. 

^en tne replacement test is called by any BIT except tne file level BIT, some 
of tne components of tne data block nave already been operationally verified. 
Tnese are not verified again by tne replacement test. Tne file level BIT, now- 
ever, always verifies every component as it is addressed. Should a component 
fail tne verification tests during a file level BIT, the appropriate component 
test is inrodiately called and tnat component tnoroug^ly cnecked out. Pro- 
ceeding in tnis manner, tne file level BIT checks tne components of an entire 
file, data block by data block until all tne faults are removed. 

5.8 HIGH-SPEED SEQUEhCER: BASIC SEQUEhCE 

The nign-speed sequencer supplies tne detailed cormBnds for the memory wafers 
to execute each system command. Each sequence is basically a variation of the 
train seouence, which is flow charted in figure 5-13. The variations of tne 
seouence are selected by tne starting address of the sequence and tne HSS sys- 
tem microprocessor status, which are supplied by tne system microprocessor.. 
Each orancn decision in the flow cnart represents a test of tne corresponding 
status bits. 

First tne HSS retrieves tne addressing information necessary to operate on a 
particular data file. The address file, corresponding to the data file, is 
retrieved from the wafers and stored in tne address buffer. The HSS-system 
microprocessor status is then checked to see which addressing sequence is 
required. Depending on tne status bits, any from one MA in one quadrant to 8 
MA*s in two quadrants can be addressed. iMhen less than a full quadrant is to 
be addressed, tne status flags also indicate which specific MA or MA pair is 
to be addressed. Full dual quadrant addressing was discussed in section 4.3.3. 
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After tne aadresslng seouence Is cnosen and executed, tne type of cata ocera- 
tion is selected by tasting enotner status flag. For read or -rite file opera- 
tions, tne EDAC mode is set up, followed by tne seouence to sMft tne data be- 
tween tne addressed and tne system interface, via tne EDAC. After conw 
pleting tne snifting seouence, tne write voltage is applied to tne mA's, wnicn 
assune tne low power write sUte. Since erase operations do not involve data 
sniftir^, tne HSS powers down tne EDAC and tnen applies tne erase voltage to 
tne addressed i^'s. After a write or erase voltage nas been apolied to tne 
addressed MA's, any MA's wnich fave finisncd tneir write or erase are addressed 
and powered down. 

wnen tne read, write, or erase seouence nas f inisned, tne mss issues a comple- 
tion interrupt to tne system microprocessor. The system microprocessor tnen 
instructs tne HSS to eltner repeat tne wnole sequence or Idle until tne next 
system cornmand is initiated. 
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Ahutract^A 5- Vainly CMOS 4.Mb naND EEFROM with high-speed 
block- page progrunming circuits and otxhip test ciraiiu for evaluating 
(he NANX>-$micmred cell is (kiqibed. This higbniefisity EEPROM hai 
WGcesslully demonstrated the ippUcabiUrr of these tccfaniqucs for micro- 
computer applications, which requra a large aonrolatUe memory sysiera 
with low power consumpdon. 



I. Introduction 

MOST modem computer systems have CPU registers, 
a random access main memory, and a sequential 
access secondary memory to achieve high performance and 
low cost per bit. The necessary features of the secondary 
memory arc large capacity, nonvolatility, and low cost per 
bit. The rotating magnetic disk, in its variations, is the 
most popular secondary' memory device and has been in 
existence for some time. Because the magnetic disk needs a 
rotating mechanism (e.g., electronic motor) and a position 
sensing mechanism, the computer system with the mag- 
netic disk systems is large and heavy. 

High-density EEPROM's are attracting much interest in 
systems designers who wish to eliminate the rotating mag- 
netic disks in portable computers and battery-powered 
laptops. Since the conventional EEPROM cell occupies a 
large area, the EEPROM*s cost per bit is high and its 
capacity is not enough to be used as secondary memory. 

Because the nand EEPROM has a very small cell 
area, it can realize high density and low cost. The nand 
EEPROM is the most promising candidate to replace the 
magnetic disk system (11-{31. 

At first, this paper introduces the NAND-structured cell 
and 4- Mb nand EEPROM and describes the novel circuit 
techniques used in this EEPROM, which are a block-page 
mode for high-speed programming and several test modes 
10 realize a highly reliable nand EEPROM [41-{71. 

Maniuchpi rtceivcd August H 1989: revised December 3. 1989. 
The authors are with the ULSl Research Center. Toshiba Corporation. 
I. IComuku. Toshiba-cho. Saiwu<iiu. tCawosaki 210. Japai). 
IEEE Log Number H934071. 




(a) (b) (c) 

Fig. 1. Top view of the NANO-structured cell (a) S£M micrograph, 
(b) Layout (c) Equivalent circuit. 



II. 4-Mb NAND EEPROM 
A. NAND Structured Cell 

Fig. l(a)-(c) shows an SEM micrograph, the layout, and 
the equivalent circuit of the NAND-structured cell for the 
4-Mb NAND EEPROM, respectively. As shown in this 
figure, this NAND-structured cell arranges eight elemental 
memory cells in series sandwiched between two select gate 
transistors. Select gate 1 (.SGI) ensures selectivity and 
select gate 2 (5C2) prevents the cell current from passing 
during a programming operation. The memory cell transis- 
tors are made in a self-aligned double-polysilicon technol- 
ogy. The Qoating gates are made of first- Oower) layer 
polysilicon. The control gates are made of second- (upper) 
layer polysilicon. Both the channel length and width of the 
memory cell transistors are 1.0 ^m. 

Fig. 2 shows how the cell size can be reduced by using a 
NAND-structured cell as compared to the conventional ceil 
(NCR-structured cell). By using 1.0-fim design rules, the 
memory transistors- of both nor- and NAND-siruaured 
cells are illustrated. The NAND-structured ceil can realize a 
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Fig. 1 Comparison of conventional Non-structured cdl with the 
NAND-structured cell. 
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Fig. 3. Endurance characteristic of elemental mcmoiy ceils of one 
NANO-siructured ceU. 
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Fig. 4. «AD retention characteristic of a dctnental memory cell of a 
NANO-stnjctured oeU. 



smaller cell area than that of the conventional cell. The 
whole area of eight NAND-stmctured cells is 4.0 Mm x 25.8 
Mm and the cell area per bit is 12.9 iin^. This is only 44% 
of the area required by a NOR-snructured cell. 

The endurance of the NAND-siructurcd cell is shown in 
Fig. 3. Bght elemental memory cells of one NAND-siruc- 
turcd ceil are measured. The threshold voltage of each 
elemental memory cell after erasing/programming is inde- 
pendent of the location in the nand structure. The differ- 
ence between the threshold voltage of the cell after the 
tenth erase/program cycle and that of the cell after the 
lOOOOth erase/program cycle is very small. But the differ- 
cnce between the threshold voltage of the cell after the first 
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Fig. 5. 4-Mb NAND EEPROM. (a) Photograph. (b>Uyout. 



cycle and that after the tenth cycle is not small. The 
warm-up of ten erase/program cycles is needed. The read ■ 
retention after the lOOOOth erase/program cycle can br^ 
guaranteed for more than ten years in cases where the "~ 
control gate voluge is below 6 J V, as shown in Fig. 4. -v^ 
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A NASD EEPROM 

Fig. 5 shows a photograph and the schematic layout of , 
the 4-Mb NAND EEPROM chip. The chip size achieved i?'! 
10.7 mm X15.3 mm. The organization is S12Kxg b. 

The memory cells arc divided into four planes. EachJ 
plane has l-Mb (1024x1024) cells, in other words 128id 
NAND-structured cells. The memory cells art divided vai^ 
512 blocks for the block-page mode. :~~ 

The circuit techniques of the 4-Mb nand EEPROM are 
described briefly. On-chip high-voltage generators can rektfi^ 
izc 5-V-only erase/program operations, because erase and S 
program arc performed by Fowler-Nordheim tunnelingl'" 
The block selector drives one pair of 512 pairs of SCI and 
SGI. The CC driver appUes 5 V/0 V and ^c«/^ct i^c 
control gates in the normal and test modes, respectively. 
CG boosters raise the control gates of the selected nand 
unit to erase/program voltages, which are generated by an 
on-chip high-voltage generator. The sense amplifier of the 
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dynamic sensing scheme ensures sensing against the small 
worsi-case read current The address transient deiecior 
(AID) generates the trigger signal of the dynamic sensing 
operation. Each bit line has a data register and a bit-line 
control circuit for block-page programming. Timers and an 
address buffer, which includes an address latch, can realize 
a self-iimed erase/program cycle which frees the CPU to* 
perform other tasks during erasing/programming. The 
R/B buffer is the output circuit of the R/B page signal 
for indicating one erase/program period and of the R /B 
block signal for indicating the whole block-page mode 
period. 

C Process and Device Technologies 

The memory cells are fabricated by a 1-^m self-aligned 
double-polysilicon technology. The timnel-oxide thickness 
is around 100 A. The imer-polysilicon dielectric is an 
oxide -nitride-oxide (ONO) stack with a capacitivc equiva- 
lent oxide thickness of 2S0 A. 

The peripheral drcuiis arc fabricated by a 2-^m triple- 
polysilicon n-well CMOS process. The gate oxide thickness 
is 400 A. The gate lengths of NMOS and PMOS are 10 
and 2.5 /im, respectively. 

The main device parameters are shown in Table I. 



in. Block-Paoe Mode 

Fig. 6 shows a block diagram of the 4-Mb nand 
EEPROM. The 4.Mb cells are split into 512 blocks. One 
block consists of 1024 nand cell units. One block is the 
least unit for crasing/programnung. 

One block-page cycle consists of one block simultaneous 
erase cycle and eight successive page program cycles. Fig, 7 
shows one block cell array and its control circuits. At first. 
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Fig. 6. Block diagram of the 4.Mb nand EEPROM. 
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Fig, 7. One block cell array and iu conirol drcuiu. 

all cells in the selected block are erased at once by apply- 
ing 17 V to all control gates of the selected block and 0 V 
to the 1024 bit lines. Electrons are injected into ail the 
floating gates of the cells. The threshold voluge of the 
erased ceil becomes the enhancement mode of approxi- 
mately 2 V. All cells in the block are erased simultane- 
ously. 

After the block erase operation is accomplished, the 
page programming staru from the source side cells, which 
are connected with the CG8 line, to the bit-line side cells, 
which are connected with the CGI line, successively, in 
order to prevent interference between programmed 
and cells being programmed. One page-program cycle con- 
sisu of a 1024-b data load sequence followed by a page 
program sequence. After the data load sequence, each of 
the bit lines is raised to programming voltages in accor- 
dance with the logic state of iu dau register. 

At firsu the programming stans at the source side cells 
which are connected with CG8. Zero volts are applied to 
the control gate of the selected cells (CCS) while 22 V are 
applied to the control gates of the unsclected cells 
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Fig. 8. Switching wavefonns of block-page mode. 

(CC1-CC7), which are closer to the bit line than the 
selected cells. These unselected cells act as pass transistors. 

In the case of zero programming, the bit line is raised 
to 22 V and electrons are emitted from the floating gate to 
the bit line by the electric field between the bit line and the 
floating gate of the selected ceil. The threshold voltage of 
the selected cell is pushed into the depletion mode of 
approximately - 3 V. 

In the case of one programming, the bit line is raised to 
11 V. But no electrons are emitted from the floating gate 
to the bit line, because the electric field between the bit 
line and the floating gate of the selected cell is insufficient 
to initiate tunneling. The threshold voltage of the selected 
cell remains at 2 V. 

After programming the cells connected with CCS, the 
programming of the cells connected with CC7 starts. Zero 
volts are applied to the control gate of the selected cell 
(CC7) and the control gate of the unselected cell (CCS), 
which is closer to the source line than the selected cell 
(CC7). Twenty two volts are applied to the control gates 
of the unselected cells (CCl-CGSl which are closer to the 
bit line than the selected cell i CC7). The bit line is raised 
to 22 V in the case of zero programming, and 11 V in the 
case of ONE programming. 

Subsequently, the programming continues from the 
source side cell to the bit-line side cell successively. Typical 
erasing time per block is 1 ms. Typical programming ume 
per page,' including the dau load sequence, is 1 ms. As a 
result, the total programming time of one block page is 
9 ms. 

Fig. 8 shows the switching wavefonns of the block-page 
.mode. . 

After CE goes down to "low," fim WE toggle triggcn a 
block erase operation. Following the erase operation, eight 
page program operations arc performed. Input data arc 
stored in data registers corresponding to WE toggles. After 
WE stays at "high" level for around 1 ms, this EEPROM 
interprets that the data load sequence has finished. Each of 
the stored data is transferred to each of the bit hnes: 1024 
bits of data are programmed to the selected 1024 cells. 

This EEPROM has the data registers, address input 
latch circuits, and timer circuit for self-timed erase/pro- 
gratn. The EEPROM also has two extra output signals: the 
R/B page output signal indicates one erase/program pe- 
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riod and the R/B block output signal indicates the whole 
block-page period. 

If the CPU host monitors these signals, the CPU is free' 
of controlling this EEPROM except for the data load 
sequence. The data registers, address input latch circuits, 
timer, and R/B output signals make interface circuits 
simple and raise CPU efficiency. 

rv. Test Mode 

Tests for a EEPROM cell, for example, the endurancel 
test and the dau retention test, are important for realizing 
a highly reUable EEPROM. But these tests are time- 
consuming. 

This 4-Mb NAND EEPROM has several test modes for^ 
shortening test time and raising test efficiency. When the 
TE of the extra control input is "high," this EEPROM is* 
in a test mode. 

A. Chip Erase and ail Block-Page Program 

When address input .40 is raised to for a positive 
voltage between 9 and 15 V while TE is -high,** the chip 
erase mode is initiated and 4-Mb cells are all erased at 
once. 

The all block-page mode is identical to the one block- 
page mode except that address input AU is raised to VuJ^^ 
whUe TE is -high.** Then, 512 blocks are block-pagc 
programmed at the same time. r."* 

The chip erase mode and the all block-page mode arc 
used to shorten the test time when carrying out tests that 
require a large number of erase/program cycles, such as 
the device endurance test. 

B. Cell Threshold Voltage Measure Mode LEX02853 

In the NAND EEPROM, the unselected ceU acts as a pass 
transistor in RExD/program operation. So, the threshold^ 
voltage of the one programmed cell must be controlled iiL"4^ 
the range between 1 and 3 V. In this section, the technique 
of measuring the threshold voltage of a one programmed 
cell is described. 

Fig. 9 shows the typical read current as a function of ' 
the control gate voltage of the selected elemental memory^ 
cell of the NAND-strucmred cell In this case, the control 
gate voltage of the unselected elemental memory cells dfj 
the NANi>.$iructured ccU are 5 V. The threshold voltage of 
the ONE programmed cell is around 2 V. The threshold; 
voltage of the zero programmed cell is around - 3 V. 

In the normal read operation, the control gate volt-^- . 
of the selected cell is 0 V, and that of the unselected cell is - 
5 V. So, the unselected cell can act as a pass transistor ' 
even if this cell is one programmed If a zero programmed 
cell is selected, the selected NAND-structured cell generates 
READ current. If a one programmed ceU is selected, the ^ 
NAND-structurcd cell does not generate read current. ^ 

Now, in the test read mode« a control gate voltage can 
be externally applied, K^-^ is applied to the control gate of . ^ 
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Fig, 10. Schmoo plot of test read mode. 



the selected cell. V^f^ is applied to the control gates of the 
unselected cells. 

^CL applied over 2 V. So, the NAND-stnictured cell 
generates read current even if the one programmed cell is 
ielected. If V^,f is below 2 V, the unselected one pro- 
grammed ceil does not act as a pass transistor. So the 
NAND-sinicturcd cell docs not generate read currcm even 
if the ZERO programmed ceil is selected. 

Fig. 10 shows a schmoo plot in the test read mode. The 
ZERO fail area shows that V^^ is smaller than the threshold 
voltage of the unselected one programmed cell. So cell 
current does not flow in spite of the ZERO progranuned cell 
selected. The one fail area shows that Vq^ is larger than 
the threshold voltage of the selected one programmed cell. 
So cell current flows in spite of the one progranuned cell 
selected. 

Thus, the threshold voltage of a one programmed cell 
can be easily evaluated from this schmoo plot. In this case, 
the threshold voltage of the one programmed cell is about 

3 V. 

By using this mode, the deviation in threshold voluge of 
a one programmed cell, between the initial state and the 
post-stressed state, can be measured. Fig. 11 shows the 
distribution of the threshold voltage of the one pro> 
grammed cell in the 4-Mb nand EEPROM. The distribu- 
tion of the threshold voltage after ten erase/program 
cycles is between 1.6 and 3.2 V. The distribution after 
10 000 erase/program cycles is between 1.2 and 2.8 V. The 
deviation between the 10th and lOOOOih cycles is so small 
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Fig. IX Switching waveforms of test erase/prognm ooode. 

that this 4-Mb EEPROM reliably performs after 10000 
erase/program cycles. 

C. External Timer and External Erase/ Program Voltage 

This nand EEPROM has two extra control inputs for 
evaluating the NAND-structured cell. While TE is "higf 
erase time and program time arc_controlled by the 
and I^„, inputs, respectively. If CE transits from "low" to 
'"high" during an erase/program cycle in a block-page 
mode, the next program cycle caxmot be initiated in the 
same block-page mode cycle. 

Switching waveforms of the test erase/program mode 
are shown in Fig. 12. 

On-chip high*voltage generators have output pads. From 
these output pads, erase/program voltages can be exter* 
nally applied variably. . 

The features of the NAND-structured cell about erase/ 
program can be evaluated by using these modes. 

V. APPLICATION LEX02854 

Fig. 13 shows floppy disks attached to a typical com- 
puter system and Fig. 14 illustrates nand EEPR0M*s 
attached to a similar system. The system with floppy disks 
needs a floppy disk drive (FDD) that has a rotating 



422 




I/O 
I devices 



CPU I * roo ;^_os_''5 ; 

I '^^"^'>"";«ES:^ interface''^ 



n>r> 



U«mei7 



riuppy 
disk , 



f 1 i \^ 



Nn>D 



/floppy 
#3 



BUS 
Arbiter 



roo :: Hoppy Disk Dmer 
OS:: 0«U Separator 



Fig. 13. Block diagram o( computer system with floppy disk*. 
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mechanism and a position sensing mechanism. An inter- 
face unit consists of FDD interface, a data separator (DS). 
an FDD controller, and CPU interface. DS extracts valid 
data from output signals of FDD. CPU interface contains 
a parallel-serial converter. Thus, this interface uni< is 
complex. 
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On the other hand* the nand EEPROM is a tiny semi* ^ 
conductor chip. The nand EEPROM requires no extra'"? 
control unit except for the CPU. The system with the ife 
nand EEPROM is simpler, lighter, and less power-con* 
suming than the system with magnetic media for a sec- 
ondary memory device. Also, the access time per bit or 
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pr;:irum time per bit is 1/lOOOth shorter than that of the 
magnetic media including the seek lime. 

To further raise the CPU utilization on this system, a 
. i-kilobyie buffer RAM and a controller unit are needed 
for interface to the nand EEPROM's. The buffer RAM is 
used for temporary storage of one block data. The con- 
troller has an address register for temporary storage of a 
selected block address and a binary counter for generating 
page and NAND*control address. By supporting these cir- 
cuits, the CPU only carries out 1-kilobyte dau loading to 
the buffer RAM during one block-page mode (as shown in 
Fig. 15). 

The NAND EEPROM needs no power to retain its stor- 
age data. The EEPROM needs only 5-V power supply. 
Therefore, the nand EEPROM is suitably for compact 
microcomputer applications, which are off-line systems 
with large storage. The memory card is an especially good 
application for the nand EEPROM, whose characteristics 
are large capacity and nonvoiatility with no power supply. 
This card can be used for personal cards of hospitals and 
banks, for an electronic camera, for the data cartridge of 
an electronic musical instrument, and for navigation sys- 
tems. 
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VI, SimMARY 

A hish-density» S-V^nly CMOS EEPROM with a 
NAND-structured cell using Fowler-Nordheim tunneling 
for programming has been realized. The block-page mode 
is adopted for high-speed programming and easy micro- 
processor interface. On-chip test circuits for test-time 
shortening and for cell characteristic evaluation realize 
highly reliable EEPROM's. The nand EEPROM has many 
applications for microcomputer systems that require small 
size and large nonvolatile storage systems with low power 
consumption. 
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